2015-02-24

Additional charts

Host zm08.stanford.edu: Total CPU

avg(user) = 5.71    avg(sys) = 0.69    avg(idle) = 90.22    avg(iowait) = 3.29    avg(nice) = 0.00    avg(irq) = 0.00    avg(softirq) = 0.08

Host zm08.stanford.edu: Individual CPU user time

avg(cpu0) = 15.73    avg(cpu1) = 15.57    avg(cpu2) = 7.97    avg(cpu3) = 7.72    avg(cpu4) = 4.96    avg(cpu5) = 6.14    avg(cpu6) = 3.79    avg(cpu7) = 3.43    avg(cpu8) = 2.67    avg(cpu9) = 4.81    avg(cpu10) = 2.19    avg(cpu11) = 4.96    avg(cpu12) = 7.55    avg(cpu13) = 6.37    avg(cpu14) = 6.13    avg(cpu15) = 3.95    avg(cpu16) = 16.90    avg(cpu17) = 3.45    avg(cpu18) = 1.88    avg(cpu19) = 4.46    avg(cpu20) = 1.37    avg(cpu21) = 2.10    avg(cpu22) = 1.16    avg(cpu23) = 1.77

Host zm08.stanford.edu: Individual CPU nice time

avg(cpu0) = 0.00    avg(cpu1) = 0.00    avg(cpu2) = 0.00    avg(cpu3) = 0.00    avg(cpu4) = 0.00    avg(cpu5) = 0.00    avg(cpu6) = 0.00    avg(cpu7) = 0.00    avg(cpu8) = 0.00    avg(cpu9) = 0.00    avg(cpu10) = 0.00    avg(cpu11) = 0.00    avg(cpu12) = 0.00    avg(cpu13) = 0.00    avg(cpu14) = 0.00    avg(cpu15) = 0.00    avg(cpu16) = 0.00    avg(cpu17) = 0.00    avg(cpu18) = 0.00    avg(cpu19) = 0.00    avg(cpu20) = 0.00    avg(cpu21) = 0.00    avg(cpu22) = 0.00    avg(cpu23) = 0.00

Host zm08.stanford.edu: Individual CPU sys time

avg(cpu0) = 1.43    avg(cpu1) = 1.51    avg(cpu2) = 0.95    avg(cpu3) = 0.96    avg(cpu4) = 0.66    avg(cpu5) = 0.78    avg(cpu6) = 0.53    avg(cpu7) = 0.50    avg(cpu8) = 0.35    avg(cpu9) = 0.55    avg(cpu10) = 0.28    avg(cpu11) = 0.51    avg(cpu12) = 0.85    avg(cpu13) = 0.79    avg(cpu14) = 1.08    avg(cpu15) = 0.83    avg(cpu16) = 1.36    avg(cpu17) = 0.59    avg(cpu18) = 0.34    avg(cpu19) = 0.55    avg(cpu20) = 0.24    avg(cpu21) = 0.38    avg(cpu22) = 0.20    avg(cpu23) = 0.26

Host zm08.stanford.edu: Individual CPU idle time

avg(cpu0) = 57.44    avg(cpu1) = 52.96    avg(cpu2) = 89.49    avg(cpu3) = 89.49    avg(cpu4) = 93.61    avg(cpu5) = 91.96    avg(cpu6) = 95.31    avg(cpu7) = 95.57    avg(cpu8) = 96.77    avg(cpu9) = 94.00    avg(cpu10) = 97.41    avg(cpu11) = 93.97    avg(cpu12) = 84.59    avg(cpu13) = 84.76    avg(cpu14) = 92.17    avg(cpu15) = 94.92    avg(cpu16) = 81.10    avg(cpu17) = 95.75    avg(cpu18) = 97.67    avg(cpu19) = 94.19    avg(cpu20) = 98.31    avg(cpu21) = 97.31    avg(cpu22) = 98.57    avg(cpu23) = 97.79

Host zm08.stanford.edu: Individual CPU iowait time

avg(cpu0) = 25.14    avg(cpu1) = 29.66    avg(cpu2) = 1.45    avg(cpu3) = 1.68    avg(cpu4) = 0.70    avg(cpu5) = 0.98    avg(cpu6) = 0.31    avg(cpu7) = 0.43    avg(cpu8) = 0.16    avg(cpu9) = 0.53    avg(cpu10) = 0.09    avg(cpu11) = 0.45    avg(cpu12) = 6.88    avg(cpu13) = 8.03    avg(cpu14) = 0.48    avg(cpu15) = 0.27    avg(cpu16) = 0.40    avg(cpu17) = 0.18    avg(cpu18) = 0.10    avg(cpu19) = 0.69    avg(cpu20) = 0.05    avg(cpu21) = 0.18    avg(cpu22) = 0.05    avg(cpu23) = 0.17

Host zm08.stanford.edu: Individual CPU irq time

avg(cpu0) = 0.00    avg(cpu1) = 0.00    avg(cpu2) = 0.00    avg(cpu3) = 0.00    avg(cpu4) = 0.00    avg(cpu5) = 0.00    avg(cpu6) = 0.00    avg(cpu7) = 0.00    avg(cpu8) = 0.00    avg(cpu9) = 0.00    avg(cpu10) = 0.00    avg(cpu11) = 0.00    avg(cpu12) = 0.00    avg(cpu13) = 0.00    avg(cpu14) = 0.00    avg(cpu15) = 0.00    avg(cpu16) = 0.00    avg(cpu17) = 0.00    avg(cpu18) = 0.00    avg(cpu19) = 0.00    avg(cpu20) = 0.00    avg(cpu21) = 0.00    avg(cpu22) = 0.00    avg(cpu23) = 0.00

Host zm08.stanford.edu: Individual CPU softirq time

avg(cpu0) = 0.25    avg(cpu1) = 0.29    avg(cpu2) = 0.12    avg(cpu3) = 0.13    avg(cpu4) = 0.07    avg(cpu5) = 0.13    avg(cpu6) = 0.06    avg(cpu7) = 0.06    avg(cpu8) = 0.03    avg(cpu9) = 0.10    avg(cpu10) = 0.02    avg(cpu11) = 0.09    avg(cpu12) = 0.13    avg(cpu13) = 0.03    avg(cpu14) = 0.12    avg(cpu15) = 0.01    avg(cpu16) = 0.23    avg(cpu17) = 0.01    avg(cpu18) = 0.01    avg(cpu19) = 0.09    avg(cpu20) = 0.01    avg(cpu21) = 0.02    avg(cpu22) = 0.01    avg(cpu23) = 0.00

Host zm08.stanford.edu: Process CPU

avg(mailbox) = 4.58    avg(mysql) = 0.73    avg(stats) = 0.00

Host zm08.stanford.edu: Process Total Memory

avg(mailbox) = 21296.10    avg(mysql) = 18153.30    avg(stats) = 0.00

Host zm08.stanford.edu: Process Resident Memory

avg(mailbox) = 8877.92    avg(mysql) = 17904.65    avg(stats) = 0.00

Host zm08.stanford.edu: Virtual Memory

avg(swapped) = 12.92    avg(free) = 672.56    avg(active) = 39558.19    avg(inactive) = 22307.61    avg(page cache) = 31876.01

Host zm08.stanford.edu: Context Switches

avg(context switches) = 10715.01

Host zm08.stanford.edu: Run/Blocked Process Queue Size, Load Average

avg(processes waiting for run time) = 2.04    avg(processes waiting for io time) = 1.19    avg(load average (1 min)) = 2.67

Host zm08.stanford.edu: Disk Utilization (Top 10 avg)

avg(sdf) = 22.18    avg(dm-0) = 17.82    avg(sdg) = 13.79    avg(dm-2) = 34.74    avg(sdl) = 22.15    avg(sdh) = 17.65    avg(dm-4) = 23.56    avg(dm-3) = 7.38    avg(sdm) = 13.81    avg(sdn) = 17.67

Host zm08.stanford.edu: Disk Throughput: /dev/sde

avg(sde read) = 0.19    avg(sde write) = 0.00

Host zm08.stanford.edu: Disk Throughput: /dev/sdf

avg(sdf read) = 770.38    avg(sdf write) = 986.95

Host zm08.stanford.edu: Disk Throughput: /dev/sda

avg(sda read) = 0.19    avg(sda write) = 0.00

Host zm08.stanford.edu: Disk Throughput: /dev/sdk

avg(sdk read) = 21.82    avg(sdk write) = 35.93

Host zm08.stanford.edu: Disk Throughput: /dev/sdt

avg(sdt read) = 0.19    avg(sdt write) = 0.00

Host zm08.stanford.edu: Disk Throughput: /dev/sdq

avg(sdq read) = 0.19    avg(sdq write) = 0.00

Host zm08.stanford.edu: Disk Throughput: /dev/sdj

avg(sdj read) = 0.33    avg(sdj write) = 167.50

Host zm08.stanford.edu: Disk Throughput: /dev/dm-0

avg(dm-0 read) = 1340.62    avg(dm-0 write) = 519.93

Host zm08.stanford.edu: Disk Throughput: /dev/sdg

avg(sdg read) = 670.73    avg(sdg write) = 258.86

Host zm08.stanford.edu: Disk Throughput: /dev/dm-1

avg(dm-1 read) = 0.27    avg(dm-1 write) = 336.96

Host zm08.stanford.edu: Disk Throughput: /dev/dm-2

avg(dm-2 read) = 1540.62    avg(dm-2 write) = 1974.83

Host zm08.stanford.edu: Disk Throughput: /dev/sdl

avg(sdl read) = 770.62    avg(sdl write) = 987.89

Host zm08.stanford.edu: Disk Throughput: /dev/sdc

avg(sdc read) = 0.19    avg(sdc write) = 0.00

Host zm08.stanford.edu: Disk Throughput: /dev/sdr

avg(sdr read) = 0.19    avg(sdr write) = 0.00

Host zm08.stanford.edu: Disk Throughput: /dev/sds

avg(sds read) = 0.19    avg(sds write) = 0.00

Host zm08.stanford.edu: Disk Throughput: /dev/sdi

avg(sdi read) = 404.24    avg(sdi write) = 591.53

Host zm08.stanford.edu: Disk Throughput: /dev/sdh

avg(sdh read) = 1072.17    avg(sdh write) = 0.24

Host zm08.stanford.edu: Disk Throughput: /dev/dm-4

avg(dm-4 read) = 2144.33    avg(dm-4 write) = 0.47

Host zm08.stanford.edu: Disk Throughput: /dev/sdb

avg(sdb read) = 0.19    avg(sdb write) = 0.00

Host zm08.stanford.edu: Disk Throughput: /dev/dm-3

avg(dm-3 read) = 808.28    avg(dm-3 write) = 1183.90

Host zm08.stanford.edu: Disk Throughput: /dev/sdm

avg(sdm read) = 670.28    avg(sdm write) = 261.07

Host zm08.stanford.edu: Disk Throughput: /dev/sdp

avg(sdp read) = 0.33    avg(sdp write) = 169.46

Host zm08.stanford.edu: Disk Throughput: /dev/sdn

avg(sdn read) = 1072.55    avg(sdn write) = 0.23

Host zm08.stanford.edu: Disk Throughput: /dev/sdu

avg(sdu read) = 0.19    avg(sdu write) = 0.00

Host zm08.stanford.edu: Disk Throughput: /dev/sdd

avg(sdd read) = 0.19    avg(sdd write) = 0.00

Host zm08.stanford.edu: Disk Throughput: /dev/sdo

avg(sdo read) = 404.43    avg(sdo write) = 592.37

Host zm08.stanford.edu: Disk IOPs: /dev/sde

avg(sde read) = 0.05    avg(sde write) = 0.00

Host zm08.stanford.edu: Disk IOPs: /dev/sdf

avg(sdf read) = 50.67    avg(sdf write) = 37.76

Host zm08.stanford.edu: Disk IOPs: /dev/sda

avg(sda read) = 0.05    avg(sda write) = 0.00

Host zm08.stanford.edu: Disk IOPs: /dev/sdk

avg(sdk read) = 0.83    avg(sdk write) = 1.71

Host zm08.stanford.edu: Disk IOPs: /dev/sdt

avg(sdt read) = 0.05    avg(sdt write) = 0.00

Host zm08.stanford.edu: Disk IOPs: /dev/sdq

avg(sdq read) = 0.05    avg(sdq write) = 0.00

Host zm08.stanford.edu: Disk IOPs: /dev/sdj

avg(sdj read) = 0.08    avg(sdj write) = 0.78

Host zm08.stanford.edu: Disk IOPs: /dev/dm-0

avg(dm-0 read) = 84.69    avg(dm-0 write) = 26.78

Host zm08.stanford.edu: Disk IOPs: /dev/sdg

avg(sdg read) = 42.39    avg(sdg write) = 13.39

Host zm08.stanford.edu: Disk IOPs: /dev/dm-1

avg(dm-1 read) = 0.07    avg(dm-1 write) = 1.56

Host zm08.stanford.edu: Disk IOPs: /dev/dm-2

avg(dm-2 read) = 101.25    avg(dm-2 write) = 75.52

Host zm08.stanford.edu: Disk IOPs: /dev/sdl

avg(sdl read) = 50.67    avg(sdl write) = 37.76

Host zm08.stanford.edu: Disk IOPs: /dev/sdc

avg(sdc read) = 0.05    avg(sdc write) = 0.00

Host zm08.stanford.edu: Disk IOPs: /dev/sdr

avg(sdr read) = 0.05    avg(sdr write) = 0.00

Host zm08.stanford.edu: Disk IOPs: /dev/sds

avg(sds read) = 0.05    avg(sds write) = 0.00

Host zm08.stanford.edu: Disk IOPs: /dev/sdi

avg(sdi read) = 6.53    avg(sdi write) = 30.14

Host zm08.stanford.edu: Disk IOPs: /dev/sdh

avg(sdh read) = 22.54    avg(sdh write) = 0.02

Host zm08.stanford.edu: Disk IOPs: /dev/dm-4

avg(dm-4 read) = 44.98    avg(dm-4 write) = 0.04

Host zm08.stanford.edu: Disk IOPs: /dev/sdb

avg(sdb read) = 0.05    avg(sdb write) = 0.00

Host zm08.stanford.edu: Disk IOPs: /dev/dm-3

avg(dm-3 read) = 12.96    avg(dm-3 write) = 60.28

Host zm08.stanford.edu: Disk IOPs: /dev/sdm

avg(sdm read) = 42.40    avg(sdm write) = 13.39

Host zm08.stanford.edu: Disk IOPs: /dev/sdp

avg(sdp read) = 0.08    avg(sdp write) = 0.78

Host zm08.stanford.edu: Disk IOPs: /dev/sdn

avg(sdn read) = 22.54    avg(sdn write) = 0.02

Host zm08.stanford.edu: Disk IOPs: /dev/sdu

avg(sdu read) = 0.05    avg(sdu write) = 0.00

Host zm08.stanford.edu: Disk IOPs: /dev/sdd

avg(sdd read) = 0.05    avg(sdd write) = 0.00

Host zm08.stanford.edu: Disk IOPs: /dev/sdo

avg(sdo read) = 6.53    avg(sdo write) = 30.14

Host zm08.stanford.edu: Disk Usage

avg(/opt/zimbra/redo-mysql) = 31.69    avg(/usr/vice/cache) = 13.22    avg(/opt/zimbra/store) = 70.00    avg(/opt/zimbra) = 67.99    avg(/opt/zimbra/hsm1) = 90.00    avg(/) = 8.01    avg(/opt/zimbra/redolog) = 23.33

Host zm08.stanford.edu: Open file descriptor count

avg(total system file descriptors open) = 7466.35    avg(mailboxd file descriptors open) = 4097.52

Host zm08.stanford.edu: Process CPU Time (Top 10 avg)

avg(tripwire) = 2181.00    avg(java) = 1871.59    avg(gzip) = 2435.33    avg(mysqld) = 528.64    avg(perl) = 40.40    avg(httpd) = 36.36    avg(mysqldump) = 145.96    avg(zmmailboxd) = 3295.38    avg(horsewatcher-fe) = 48.03    avg(zmcompresslogs) = 1136.43

Host zm08.stanford.edu: SOAP: Invocation Count: Summary (Top 10 max)

max(MsgActionRequest.move) = 1091.00    max(SearchRequest) = 160.00    max(GetContactsRequest) = 1080.00    max(MsgActionRequest.delete) = 1081.00    max(NoOpRequest) = 406.00    max(WaitSetRequest) = 247.00    max(AddMsgRequest) = 174.00    max(GetFolderRequest) = 504.00    max(SyncRequest) = 504.00    max(GetAppointmentRequest) = 1839.00

Host zm08.stanford.edu: SOAP: Average Call Duration: Summary (Top 10 avg)

avg(DeleteAccountRequest) = 27275.00    avg(BackupQueryRequest) = 823.60    avg(GetAggregateQuotaUsageOnServerRequest) = 1676.00    avg(CreateAppointmentExceptionRequest) = 870.38    avg(FolderActionRequest.delete) = 658.90    avg(SendInviteReplyRequest) = 489.39    avg(ApplyFilterRulesRequest) = 127093.50    avg(ModifyPrefsRequest) = 948.36    avg(FolderActionRequest.empty) = 2152.41    avg(GetQuotaUsageRequest) = 2544.00

Host zm08.stanford.edu: MySQL: InnoDB Buffer Pool Pages

avg(pages read) = 398229581.22    avg(pages written) = 145635706.28

Host zm08.stanford.edu: MySQL: InnoDB Buffer Pool Hit Rate

avg(buffer pool hit rate) = 999.36

Host zm08.stanford.edu: MySQL: Database Connections In Use

avg(connections in use) = 0.60

Host zm08.stanford.edu: MySQL: Tables Open/Opened

avg(open_tables) = 1188.83    avg(opened_tables (cumulative)) = 480500.76

Host zm08.stanford.edu: MySQL: Total Slow Queries Count

avg(slow queries (cumulative)) = 1406240.97

Host zm08.stanford.edu: Mailboxd: Connection Pool Get Latency

avg(mysql) = 0.18    avg(ldap) = 0.16

Host zm08.stanford.edu: Mailboxd: Lucene IO

avg(Index writes) = 92.25    avg(Index reads) = 272.28

Host zm08.stanford.edu: Mailboxd: LMTP Delivery Throughput

avg(received) = 145.56    avg(delivered) = 276.73

Host zm08.stanford.edu: Mailboxd: LMTP Delivery Rate

avg(received msgs) = 1.22    avg(delivered msgs) = 1.52    avg(received x rcpts) = 1.52

Host zm08.stanford.edu: Mailboxd: Mailbox Add Rate (Delivery Rate)

avg(mailbox add rate) = 1.93

Host zm08.stanford.edu: Mailboxd: Mailbox Add Latency (Delivery Speed)

avg(mailbox add latency) = 60.44

Host zm08.stanford.edu: Mailboxd: Request Rate by Client Protocol

avg(SOAP) = 15.92    avg(IMAP) = 88.36    avg(POP) = 3.40

Host zm08.stanford.edu: Mailboxd: Response Time by Client Protocol

avg(SOAP) = 242.85    avg(IMAP) = 23.75    avg(POP) = 59.03

Host zm08.stanford.edu: Mailboxd: Active Connections by Client Protocol

avg(IMAP) = 189.68    avg(IMAP SSL) = 746.05    avg(POP) = 0.10    avg(POP SSL) = 1.30    avg(SOAP sessions) = 863.85

Host zm08.stanford.edu: Mailboxd: Mailbox Get Count

avg(mailbox get count) = 6112.26

Host zm08.stanford.edu: Mailboxd: Mailbox Get Latency

avg(mailbox get latency) = 0.01

Host zm08.stanford.edu: Mailboxd: Mailbox Cache Hit Rate

avg(mailbox cache hit rate) = 99.96

Host zm08.stanford.edu: Mailboxd: Mailbox Item/Blob Cache Hit Rate

avg(msg blob cache) = 41.65    avg(item metadata cache) = 32.16

Host zm08.stanford.edu: Mailboxd: Minor Garbage Collection Time

avg(minor GC time) = 0.70

Host zm08.stanford.edu: Mailboxd: Major Garbage Collection Time

avg(major GC time) = 0.04

Host zm08.stanford.edu: Mailboxd: Minor Garbage Collection Count

avg(minor GC count) = 10.80

Host zm08.stanford.edu: Mailboxd: Major Garbage Collection Count

avg(major GC count) = 0.03

Host zm08.stanford.edu: Mailboxd: JVM Heap Used

avg(total) = 4738.60    avg(PS Old Gen) = 0.00    avg(CMS Old Gen) = 3857.32    avg(Tenured Gen) = 0.00    avg(Train Gen) = 0.00    avg(PS Eden Space) = 0.00    avg(PS Survivor Space) = 0.00    avg(Par Eden Space) = 780.67    avg(Par Survivor Space) = 100.60    avg(Eden Space) = 0.00    avg(Survivor Space) = 0.00

Host zm08.stanford.edu: Mailboxd: JVM Heap Free

avg(total free) = 3088.71    avg(PS Old Gen free) = 0.00    avg(CMS Old Gen free) = 2163.68    avg(Tenured Gen free) = 0.00    avg(Train Gen free) = 0.00    avg(PS Eden Space free) = 0.00    avg(PS Survivor Space free) = 0.00    avg(Par Eden Space free) = 824.95    avg(Par Survivor Space free) = 100.08    avg(Eden Space free) = 0.00    avg(Survivor Space free) = 0.00

Host zm08.stanford.edu: Mailboxd: JVM Permanent Generation and Code Cache

avg(PS Perm Gen) = 0.00    avg(Perm Gen) = 0.00    avg(CMS Perm Gen) = 136.46    avg(Code Cache) = 40.73    avg(Perm Gen [shared-ro]) = 0.00    avg(Perm Gen [shared-rw]) = 0.00    avg(PS Perm Gen free) = 0.00    avg(Perm Gen free) = 0.00    avg(CMS Perm Gen free) = 106.98    avg(Code Cache free) = 5.83    avg(Perm Gen [shared-ro] free) = 0.00    avg(Perm Gen [shared-rw] free) = 0.00

Host zm08.stanford.edu: convertd: CPU time used

avg(total cputime) = 0.34    avg(user time) = 0.31    avg(system time) = 0.02

Host zm08.stanford.edu: convertd: Resident Memory

avg(convertd rss) = 205.47

Host zm08.stanford.edu: convertd: Processes and Threads

avg(processes) = 6.07    avg(threads) = 6.08

No data available for the following charts:

Host zm08.stanford.edu: Swap Activity Host zm08.stanford.edu: Disk Throughput: /dev/dm-7 Host zm08.stanford.edu: Disk Throughput: /dev/sdx Host zm08.stanford.edu: Disk Throughput: /dev/sdab Host zm08.stanford.edu: Disk Throughput: /dev/sdaa Host zm08.stanford.edu: Disk Throughput: /dev/sdad Host zm08.stanford.edu: Disk Throughput: /dev/sdag Host zm08.stanford.edu: Disk Throughput: /dev/sdaf Host zm08.stanford.edu: Disk Throughput: /dev/sdy Host zm08.stanford.edu: Disk Throughput: /dev/sdv Host zm08.stanford.edu: Disk Throughput: /dev/sdac Host zm08.stanford.edu: Disk Throughput: /dev/dm-5 Host zm08.stanford.edu: Disk Throughput: /dev/sdw Host zm08.stanford.edu: Disk Throughput: /dev/sdae Host zm08.stanford.edu: Disk Throughput: /dev/dm-6 Host zm08.stanford.edu: Disk Throughput: /dev/sdz Host zm08.stanford.edu: Disk IOPs: /dev/dm-7 Host zm08.stanford.edu: Disk IOPs: /dev/sdx Host zm08.stanford.edu: Disk IOPs: /dev/sdab Host zm08.stanford.edu: Disk IOPs: /dev/sdaa Host zm08.stanford.edu: Disk IOPs: /dev/sdad Host zm08.stanford.edu: Disk IOPs: /dev/sdag Host zm08.stanford.edu: Disk IOPs: /dev/sdaf Host zm08.stanford.edu: Disk IOPs: /dev/sdy Host zm08.stanford.edu: Disk IOPs: /dev/sdv Host zm08.stanford.edu: Disk IOPs: /dev/sdac Host zm08.stanford.edu: Disk IOPs: /dev/dm-5 Host zm08.stanford.edu: Disk IOPs: /dev/sdw Host zm08.stanford.edu: Disk IOPs: /dev/sdae Host zm08.stanford.edu: Disk IOPs: /dev/dm-6 Host zm08.stanford.edu: Disk IOPs: /dev/sdz Host zm08.stanford.edu: Mailboxd: Dirty Lucene Index Writers Host zm08.stanford.edu: Mailboxd: Lucene IndexWriterCache Hitrate Host zm08.stanford.edu: nginx: CPU time used Host zm08.stanford.edu: nginx: Resident Memory Host zm08.stanford.edu: nginx: Processes and Threads

Summary