Ben-600 - Delivery metrics ============================ Average Delivery Times During Setup (caches cold) --------------------------------------------------- Direct to Origin: Fields: Mean Max Min Origin: 0.00117829 4.834 0.000 Direct to Midtier: Fields: Mean Max Min Origin: 0.00125514 2.868 0.036 Midtier: 0.00143373 5.911 0.000 One Edge node Live (test1) Fields: Mean Max Min Req count Origin: 0.00060664,0.75500000,0.02400000,211 Midtier: 0.00087204,0.88300000,0.09800000,211 Edge1: 0.01962736,9.90600000,0.23400000,212 Test 2 (One edge node live, cache warm from previous playout). Fields: Mean Max Min Req count Origin: 0.07100000,0.07100000,0.07100000,1 Midtier: 0.12300000,0.12300000,0.12300000,1 Edge1: 0.00000472,3.47200000,0.00000000,212 Test 3 (Edge 2 cold, mid-tier warm) Fields: Mean Max Min Req count Origin: 0.00000000,0.00000000,0.00000000,101 Midtier: 0.00723113,4.68100000,0.01100000,212 Edge1: -nan, 0.00000000,9999.00000000,0 Edge2: 0.01183491,5.75600000,0.57500000,212 Dispositions Midtier CACHE_HIT: 111 Midtier CACHE_REVALIDATED: 101 Edge 2 CACHE_MISS: 212 Formal Tests Begin ---------------------- Test 5 (Direct to Origin) Origin: 0.00295775,2.32200000,0.00000000,213 Bitrates/counts Origin #$ grep ".ts" test5.log | grep -o -P "Big_Buck_Bunny_([0-9]+)_" | cut -d_ -f4 | uniq -c 5 1024 208 2048 Test 6 (Direct to midtier, coldcache) Clearing the cache: # find /usr/share/nginx/streamingcache/ -type f -exec rm {} \; Origin: 0.00082160,0.54100000,0.02300000,213 Midtier: 0.00117840,3.91100000,0.09500000,213 Midtier dispositions # grep ".ts" test6.log | awk -F'\t' '{print $12}' | sort | uniq -c 213 CACHE_MISS Bitrates/Counts Origin: $ grep ".ts" test6.log | grep -o -P "Big_Buck_Bunny_([0-9]+)_" | cut -d_ -f4 | uniq -c 1 1024 212 512 Midtier: # grep ".ts" test6.log | grep -o -P "Big_Buck_Bunny_([0-9]+)_" | cut -d_ -f4 | uniq -c 1 1024 212 512 Test 7 (Direct to midtier, warmcache) Warming mid-tier cache: for br in 512 1024 2048; do for num in {0..10}; do curl -H "Host: cix7cricsvweeu6k.onion" -o /dev/null "http://127.0.0.1/Big_Buck_Bunny-HLS/Big_Buck_Bunny_${br}_0000${num}.ts"; done; for num in {11..99}; do curl -H "Host: cix7cricsvweeu6k.onion" -o /dev/null "http://127.0.0.1/Big_Buck_Bunny-HLS/Big_Buck_Bunny_${br}_000${num}.ts"; done; for num in {100..211}; do curl -H "Host: cix7cricsvweeu6k.onion" -o /dev/null "http://127.0.0.1/Big_Buck_Bunny-HLS/Big_Buck_Bunny_${br}_00${num}.ts"; done; done Origin: 0.17400000,0.17400000,0.17400000,1 Midtier: 0.00000000,2.65500000,0.00000000,249 Midtier dispositions # grep ".ts" test7.log | awk -F'\t' '{print $12}' | sort | uniq -c 248 CACHE_HIT 1 CACHE_MISS Bitrates/Counts Origin: $ grep ".ts" test7.log | grep -o -P "Big_Buck_Bunny_([0-9]+)_" | cut -d_ -f4 | uniq -c 1 2048 Midtier: # grep ".ts" test7.log | grep -o -P "Big_Buck_Bunny_([0-9]+)_" | cut -d_ -f4 | uniq -c 9 1024 7 2048 4 1024 43 2048 6 1024 7 512 17 1024 3 2048 7 1024 47 512 29 1024 13 512 6 1024 12 2048 5 1024 8 2048 2 1024 13 512 5 1024 6 2048 Test 8 (Cold edge, warm midtier) (all requests went via Edge2) Delivery Durations Origin: - Midtier: 0.00708837,4.59200000,0.00200000,215 Edge2: 0.01494884,6.82800000,1.25000000,215 Dispositions Midtier: 215 CACHE_HIT Edge2: 215 CACHE_MISS Bitrates Midtier: 23 1024 188 512 4 1024 Edge: 23 1024 188 512 4 1024 Test 9 (Warm edge, Warm midtier) Delivery Durations Origin: 0.00000000,0.00000000,0.00000000,1 Midtier: 0.65700000,0.65700000,0.65700000,1 Edge 2: 0.00000000,2.93700000,0.00000000,226 Dispositions Midtier: 1 CACHE_REVALIDATED Edge: 225 CACHE_HIT 1 CACHE_MISS Bitrates Midtier: 1 512 Edge: 9 1024 23 512 14 1024 39 2048 14 1024 88 512 4 1024 3 2048 23 1024 9 2048 Test 10 (Cold edge, cold midtier) Delivery Durations Origin: 0.00050935,0.43800000,0.02100000,214 Midtier: 0.00625234,5.43000000,0.08400000,214 Edge 2: 0.02064019,9.29100000,1.26700000,214 Dispositions: Midtier: 214 CACHE_MISS Edge: 214 CACHE_MISS Bitrates Midtier: 3 1024 211 512 Edge: 3 1024 211 512 Notes: Stream was mostly watchable, but jittery with artifacts at times. Occasional freezes of <= 1 second Test 11 (Two players, same stream, 1 edge node - Edge 2, cache warmish) Delivery Durations Origin: 0.00000000,0.20800000,0.00000000,222 Midtier: 0.00225413,5.00200000,0.00500000,303 Edge 2: 0.00000000,8.47100000,0.00000000,429 Dispositions Midtier: 81 CACHE_HIT 19 CACHE_MISS 203 CACHE_REVALIDATED Edge: 284 CACHE_EXPIRED 126 CACHE_HIT 19 CACHE_MISS Bitrates Origin: 5 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 177 512 3 1024 1 512 1 1024 2 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 2 1024 15 512 Midtier: 5 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 258 512 3 1024 1 512 1 1024 2 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 2 1024 15 512 Edge 2: 19 1024 1 512 2 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 353 512 1 1024 1 512 2 1024 1 512 1 1024 2 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 2 1024 30 512 Note: The versions in the edge cache had expired, so initial requests resulted in reacquiring from upstream. Player 2 was started after player 1, but ended up ahead in the stream because player 2 occasionally got stuck waiting for chunks. This position then reversed once Player 2 had to wait for the initial requests. Test 12 (Two players, same stream, 1 edge node - Edge 2, cache warm) Delivery Durations Origin: 0.00068041,0.42800000,0.00000000,194 Midtier: 0.00372165,3.77300000,0.07700000,194 Edge 2: 0.00000000,4.19700000,0.00000000,430 Dispositions Midtier: 186 CACHE_MISS 8 CACHE_REVALIDATED Edge 2: 8 CACHE_EXPIRED 236 CACHE_HIT 186 CACHE_MISS Bitrates Origin: 9 512 166 1024 6 2048 13 1024 Midtier: 9 512 166 1024 6 2048 13 1024 Edge 2: 1 1024 3 512 2 1024 1 512 3 1024 1 512 2 1024 1 512 1 1024 2 512 3 1024 1 512 3 1024 1 512 1 1024 1 512 2 1024 1 512 2 1024 1 512 2 1024 1 512 2 1024 1 512 2 1024 1 512 3 1024 1 512 1 1024 1 512 1 1024 1 512 3 1024 1 512 2 1024 2 512 1 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 2 1024 2 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 2 1024 1 512 4 1024 1 512 2 1024 1 512 1 1024 1 512 3 1024 1 512 1 1024 2 512 1 1024 1 512 1 1024 1 512 2 1024 1 512 2 1024 1 512 2 1024 1 512 3 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 2 1024 1 512 2 1024 2 512 1 1024 2 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 2 512 2 1024 1 512 1 1024 1 512 3 1024 1 512 1 1024 1 512 2 1024 1 512 2 1024 2 512 2 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 3 1024 1 512 2 1024 1 512 3 1024 1 512 1 1024 2 512 1 1024 1 512 1 1024 1 512 2 1024 1 512 2 1024 1 512 3 1024 1 512 2 1024 1 512 1 1024 1 512 1 1024 1 512 3 1024 1 512 1 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 1 1024 1 512 2 1024 1 512 2 1024 1 512 3 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 18 512 3 1024 1 512 3 1024 1 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 2 1024 1 512 2 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 1 2048 2 512 1 2048 2 512 1 2048 1 512 1 2048 2 512 1 2048 1 512 1 1024 1 2048 1 512 1 1024 1 512 1 1024 2 512 1 1024 2 512 1 1024 1 512 1 1024 2 512 1 1024 2 512 1 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 1 1024 58 512 Note: Player 1 started first. Again, player 2 got ahead, by nearly 3 minutes. Might suggest an issue on player 1 as much as with delivery Edge2 isn't allowed to revalidate, it has to reacquire. So now, repeating tests 10-12 but using Edge1 which can revalidate. Test 13 (Cold edge2, cold midtier,1 player) Delivery Durations: Origin:0.00043192,0.50600000,0.01500000,213 Midtier:0.00073709,1.30200000,0.08700000,213 Edge 1:0.01363850,9.00500000,0.44400000,213 Dispositions: Midtier: 213 CACHE_MISS Edge 1: 213 CACHE_MISS Bitrates: Midtier: 3 1024 210 512 Edge 1: 3 1024 210 512 Test 14 (Two players, same stream, 1 edge node - Edge 1, cache warmish) Delivery Durations Origin: 0.00987500,0.76100000,0.09600000,16 Midtier: 0.01562500,0.87200000,0.16000000,16 Edge 1: 0.01037037,9.94300000,0.00000000,432 Dispositions: Midtier: 16 CACHE_MISS Edge 1: 416 CACHE_HIT 16 CACHE_MISS Bitrates: Midtier: 4 1024 2 512 10 1024 Edge 1: 7 1024 1 512 1 1024 391 512 2 1024 1 512 1 1024 13 512 1 1024 1 512 1 1024 7 512 5 1024 Artificially warming the cache for all segments for the next test Test 15 (Two players, same stream, Edge 1 cache artifically warmed) Delivery Duration Origin: 0.00000000,0.16900000,0.00000000,218 Midtier: 0.00027016,1.35800000,0.00000000,248 Edge 1: 0.00000000,8.87800000,0.00000000,426 Dispositions: Midtier: 30 CACHE_HIT 1 CACHE_MISS 217 CACHE_REVALIDATED Edge 1: 178 CACHE_HIT 1 CACHE_MISS 247 CACHE_REVALIDATED Bitrates: Midtier: 1 1024 4 512 1 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 1 1024 3 512 1 1024 229 512 Edge 1: 1 1024 5 512 1 1024 1 512 1 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 2 1024 1 512 1 1024 1 512 1 1024 2 512 1 1024 400 512 Note: Players stayed in sync far better. There were a lot more revalidations than expected - partly because the warm took so long. Rerunning test as test16 Test 16 (Repeat of test 15) Delivery Durations Origin: 0.00000000,0.00000000,0.00000000,146 Midtier: 0.00039041,0.31200000,0.03500000,146 Edge 1: 0.00000000,5.49600000,0.00000000,580 Cache Dispositions: Midtier: 146 CACHE_REVALIDATED Edge 1: 434 CACHE_HIT 146 CACHE_REVALIDATED Bitrates: Midtier: 3 1024 2 512 141 1024 Edge 1: 9 1024 1 512 2 1024 1 512 2 1024 1 512 2 1024 3 512 1 1024 373 512 7 1024 3 2048 7 512 140 1024 1 512 2 1024 25 512 As delivery is still a bit shakey, I'm going to skip the linear tests for now and look at the impact of increasing segment length to 10 seconds.