Ben-600 - Delivery metrics - 10s fragments @480p ================================================== Commands --------- Get log lines for latest test Origin: tail -n $((`wc -l /var/log/nginx/access.log | cut -d\ -f1` - `grep '"GET /ENDTEST' -n /var/log/nginx/access.log | cut -d: -f1 | tail -n2 | head -n1`)) /var/log/nginx/access.log | grep streamingtest Midtier: tail -n $((`wc -l /var/log/nginx/access.log | cut -d\ -f1` - `grep '"GET /ENDTEST' -n /var/log/nginx/access.log | cut -d: -f1 | tail -n2 | head -n1`)) /var/log/nginx/access.log | grep cix7cricsvweeu6k Edge: tail -n $((`wc -l /var/log/nginx/access.log | cut -d\ -f1` - `grep '"GET /ENDTEST' -n /var/log/nginx/access.log | cut -d: -f1 | tail -n2 | head -n1`)) /var/log/nginx/access.log | grep f5jayrbaz7nmtyyr Calculate duration stats cat test3.log | grep ".ts" | grep "jwplayer.flash.swf" | awk -F '\t' 'BEGIN{tot=0; count=0; min=9999; max=0;}{tot=+$13;count+=1; if ($13 > max) max=$13 endif; if ($13 < min) min=$13 endif;}END{avg=tot/count; printf "%.8f,%.8f,%.8f,%d\n",avg,max,min,count}' Disposition counts grep ".ts" test3.log | awk -F'\t' '{print $12}' | sort | uniq -c Extracting requested bitrates grep ".ts" test5.log | grep -o -P "Big_Buck_Bunny_480p_([0-9]+)_" | cut -d_ -f5 | uniq -c Notes ------ Test ends are marked in the logs by running ben@milleniumfalcon:~$ GET -Ssed http://f5jayrbaz7nmtyyr.onion/ENDTEST Tests ------ Test 27 (Direct to Origin) Delivery Durations Origin: 0.00937097,4.55400000,0.45100000,62 Bitrates Origin: 3 1024 59 2048 Test 28 (Direct to midtier, coldcache) Delivery Durations Origin: 0.00154839,2.16500000,0.10500000,93 Midtier: 0.00210000,9.72600000,0.01700000,100 Dispositions Midtier: 7 CACHE_HIT 93 CACHE_MISS Bitrates Origin: 1 1024 4 512 2 1024 4 2048 1 1024 3 512 2 1024 3 512 2 1024 1 512 3 1024 1 2048 1 1024 3 2048 5 1024 2 2048 4 1024 2 2048 3 1024 2 512 3 1024 2 512 2 1024 1 2048 1 1024 5 512 2 1024 3 2048 3 1024 1 2048 2 1024 11 2048 2 1024 1 2048 3 1024 2 2048 Midtier: 1 1024 4 512 2 1024 4 2048 1 1024 3 512 2 1024 3 512 2 1024 2 512 3 1024 1 2048 2 1024 3 2048 5 1024 2 2048 4 1024 2 2048 4 1024 2 512 3 1024 2 512 2 1024 1 2048 2 1024 5 512 2 1024 3 2048 3 1024 1 2048 3 1024 11 2048 2 1024 2 2048 4 1024 2 2048 Note: The CACHE_HITs were unexpected. Looking at the logs, the player requested a few segments for some reason # grep Big_Buck_Bunny_480p_2048_00054.ts test28.log 127.0.0.1 - - [18/Dec/2015:16:34:28 +0000] "GET /Big_Buck_Bunny-HLS-10sec/Big_Buck_Bunny_480p_2048_00054.ts HTTP/1.1" 200 4435466 "http://ssl.p.jwpcdn.com/player/v/7.2.4/jwplayer.flash.swf" "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:39.0) Gecko/20100101 Firefox/39.0" "-" "cix7cricsvweeu6k.onion:8091" CACHE_MISS 6.014 - 127.0.0.1 - - [18/Dec/2015:16:34:40 +0000] "GET /Big_Buck_Bunny-HLS-10sec/Big_Buck_Bunny_480p_2048_00054.ts HTTP/1.1" 206 2330678 "http://ssl.p.jwpcdn.com/player/v/7.2.4/jwplayer.flash.swf" "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:39.0) Gecko/20100101 Firefox/39.0" "-" "cix7cricsvweeu6k.onion:8091" CACHE_HIT 2.090 - Test 29 (Direct to midtier, warmcache) Delivery Durations Origin: 0.00418750,2.15900000,0.10000000,32 Midtier: 0.00000000,9.94200000,0.00000000,84 Dispositions Midtier: 52 CACHE_HIT 32 CACHE_MISS Bitrates Origin: 2 1024 3 2048 5 512 4 2048 3 1024 3 2048 1 1024 1 2048 8 1024 2 2048 Midtier: 1 1024 4 512 5 1024 3 2048 5 1024 2 2048 5 1024 5 512 2 1024 2 2048 5 1024 2 2048 9 1024 4 2048 4 1024 3 2048 5 1024 3 2048 3 1024 2 2048 6 1024 4 2048 Note: Occasional playback issues towards the end. Test 30 (Cold edge, warm midtier) Delivery Durations Origin: 0.00246809,0.41600000,0.09400000,47 Midtier: 0.00250000,7.33800000,0.00000000,70 Edge 1: 0.01045833,9.81200000,0.00000000,72 Dispositions Midtier: 23 CACHE_HIT 47 CACHE_MISS Edge 1: 2 CACHE_HIT 70 CACHE_MISS Bitrates Origin: 32 512 2 2048 13 512 Midtier: 3 1024 21 512 1 1024 5 512 2 1024 11 512 3 1024 2 2048 2 1024 12 512 3 1024 5 512 Edge 1: 3 1024 21 512 1 1024 6 512 2 1024 12 512 3 1024 2 2048 2 1024 12 512 3 1024 5 512 Test 31 (Warm edge, Warm midtier) Delivery Durations Origin: 0.01686667,1.34700000,0.13100000,15 Midtier: 0.50046667,7.50700000,0.19000000,15 Edge 1: 0.00000000,9.36400000,0.00000000,70 Dispositions: Midtier: 15 CACHE_MISS Edge 1: 55 CACHE_HIT 15 CACHE_MISS Bitrates: Origin: 1 1024 1 2048 4 1024 3 2048 5 1024 1 2048 Midtier: 1 1024 1 2048 4 1024 3 2048 5 1024 1 2048 Edge 1: 12 1024 1 2048 6 1024 1 2048 2 1024 1 2048 21 1024 1 2048 1 1024 1 2048 13 1024 2 2048 1 1024 1 2048 6 1024 Test 32 (Cold edge, cold midtier) Delivery Durations Origin: 0.00198667,0.55800000,0.09500000,75 Midtier: 0.00272368,9.67400000,0.15200000,76 Edge 1: 0.03484416,9.96800000,0.00000000,77 Dispositions Midtier: 1 CACHE_HIT 75 CACHE_MISS Edge 1: 1 CACHE_HIT 76 CACHE_MISS Requested Bit rates Origin: 5 1024 3 2048 5 1024 3 2048 1 1024 5 512 8 1024 3 2048 5 1024 1 2048 11 1024 1 2048 7 512 7 1024 2 2048 8 1024 Midtier: 5 1024 3 2048 5 1024 3 2048 1 1024 5 512 8 1024 3 2048 5 1024 1 2048 10 1024 1 2048 2 1024 7 512 7 1024 2 2048 8 1024 Edge 1: 5 1024 3 2048 5 1024 3 2048 1 1024 5 512 8 1024 3 2048 6 1024 1 2048 10 1024 1 2048 2 1024 7 512 7 1024 2 2048 8 1024 Test 33 (Artificially warmed Edge 1) Delivery Durations Origin: 0.00000000,0.00000000,0.00000000,12 Midtier: 0.01025000,0.51300000,0.10700000,12 Edge 1: 0.00000000,9.58800000,0.00000000,60 Dispositions Midtier: 12 CACHE_REVALIDATED Edge 1: 48 CACHE_HIT 1 CACHE_MISS 11 CACHE_REVALIDATED Bitrates Origin: 12 1024 Midtier: 12 1024 Edge 1: 60 1024 Test 34 (Artificially warmed Edge 1 - repeat of Test 33) Delivery Durations Origin: 0.00000000,0.21400000,0.00000000,28 Midtier: 0.00732143,0.42000000,0.17500000,28 Edge 1: 0.03919672,9.94200000,0.00000000,61 Dispositions Midtier: 1 CACHE_MISS 27 CACHE_REVALIDATED Edge 1: 33 CACHE_HIT 1 CACHE_MISS 27 CACHE_REVALIDATED Bitrates Origin: 28 512 Midtier: 28 512 Edge 1: 33 1024 28 512 Test 35 - Two players, cold edge Delivery Durations Origin: 0.00165000,0.65200000,0.10600000,80 Midtier: 0.00192381,9.93500000,0.00000000,105 Edge 1: 0.02429457,9.90600000,0.00000000,129 Dispositions Midtier: 25 CACHE_HIT 80 CACHE_MISS Edge 1: 24 CACHE_HIT 105 CACHE_MISS Bitrates Origin: 1 1024 36 512 9 1024 20 512 1 1024 3 512 1 1024 1 512 8 1024 Midtier: 1 1024 1 512 1 1024 59 512 8 1024 1 512 1 1024 19 512 1 1024 4 512 9 1024 Edge 1: 1 1024 1 512 1 1024 63 512 1 1024 1 512 1 1024 1 512 2 1024 2 512 1 1024 1 512 1 1024 1 512 1 1024 2 512 1 1024 2 512 1 1024 28 512 1 1024 6 512 9 1024 Notes: The second player ended up around 2 minutes ahead of the first. Test 36 - Two players, warm edge Delivery Durations Origin: 0.00307018,1.76200000,0.11100000,57 Midtier: 0.00411667,9.84300000,0.17600000,60 Edge 1: 0.01737059,9.50600000,0.00000000,170 Dispositions Midtier: 3 CACHE_HIT 57 CACHE_MISS Edge 1: 110 CACHE_HIT 60 CACHE_MISS Bitrates Origin: 12 1024 1 2048 4 1024 1 2048 13 1024 7 2048 5 1024 1 2048 1 1024 1 2048 1 1024 10 2048 Midtier: 12 1024 1 2048 5 1024 1 2048 12 1024 8 2048 5 1024 1 2048 1 1024 1 2048 1 1024 12 2048 Edge 1: 17 1024 1 512 2 1024 2 512 9 1024 1 2048 6 1024 2 512 1 1024 2 512 1 2048 3 512 4 1024 2 512 1 1024 1 512 1 1024 1 512 1 1024 8 512 1 1024 4 512 1 1024 1 512 1 1024 3 512 2 1024 1 512 2 1024 7 512 1 1024 2 512 2 1024 2 512 1 2048 2 1024 1 2048 9 1024 1 2048 1 1024 1 2048 1 1024 1 2048 1 1024 1 2048 1 1024 1 2048 1 1024 1 2048 9 1024 1 2048 4 1024 1 2048 1 1024 1 2048 3 1024 1 2048 1 1024 1 2048 1 1024 5 2048 3 1024 1 2048 2 1024 1 2048 3 1024 1 2048 1 512 2 1024 1 512 1 1024 1 512 4 2048 Test 37 - Two players, warm edge Decided to re-run Test 36 as it shows an improvement in the bitrates used. Want to see whether we see a further improvement by playing again Delivery Durations Origin: 0.00843243,1.12300000,0.10400000,37 Midtier: 0.33437838,9.82100000,0.16300000,37 Edge 1: 0.00000000,9.96200000,0.00000000,166 Dispositions Midtier: 37 CACHE_MISS Edge 1: 129 CACHE_HIT 37 CACHE_MISS Bitrates Origin: 15 2048 1 1024 3 2048 1 1024 17 2048 Midtier: 15 2048 1 1024 2 2048 1 1024 18 2048 Edge 1: 1 1024 4 512 4 1024 1 512 1 2048 2 512 1 2048 1 512 1 1024 3 2048 1 1024 1 512 1 2048 1 512 1 2048 1 512 1 2048 1 512 1 2048 1 1024 1 2048 3 512 1 2048 1 1024 1 2048 2 512 1 2048 2 512 1 2048 1 512 2 2048 3 512 1 2048 1 1024 1 512 1 1024 1 512 2 1024 1 512 2 1024 2 512 1 2048 2 512 1 2048 1 1024 2 512 1 2048 1 512 1 1024 1 2048 2 512 1 2048 2 512 1 2048 1 1024 2 512 1 2048 1 512 1 2048 1 1024 3 2048 2 512 1 2048 2 512 1 2048 1 512 1 2048 1 512 1 1024 1 2048 1 1024 2 512 2 1024 1 2048 2 1024 1 2048 1 512 1 2048 1 512 2 2048 1 512 2 2048 1 1024 2 512 1 2048 2 512 1 1024 1 512 1 2048 3 512 1 2048 1 1024 2 512 1 1024 1 2048 1 1024 6 2048 1 512 1 2048 1 512 1 2048 2 512 1 2048 1 512 1 1024 1 2048 2 512 1 2048 2 512 1 1024 2 512 1 2048 2 1024 1 512 4 2048 Feels like a further improvement, lets run again Test 38 - Two players, warm edge Delivery Durations Origin: 0.05725000,0.34100000,0.20200000,4 Midtier: 1.98320000,9.91600000,12.88300000,5 Edge 1: 0.00000000,9.73400000,0.00000000,161 Dispositions Midtier: 1 CACHE_HIT 4 CACHE_MISS Edge 1: 156 CACHE_HIT 5 CACHE_MISS Bitrates Origin: 3 2048 1 1024 Midtier: 4 2048 1 1024 Edge 1: 2 1024 4 2048 1 1024 1 2048 2 1024 2 2048 1 1024 3 512 1 2048 2 1024 3 2048 3 512 1 2048 3 1024 11 2048 1 1024 1 2048 1 1024 3 2048 2 1024 1 2048 1 1024 1 2048 8 1024 1 2048 2 1024 1 2048 1 1024 1 2048 1 1024 3 512 1 2048 4 1024 1 2048 1 512 2 2048 1 512 1 2048 1 512 1 1024 1 2048 1 512 1 1024 2 2048 1 1024 13 2048 3 1024 20 2048 1 1024 1 2048 1 1024 1 2048 3 512 1 2048 2 1024 27 2048