Network connectivity between SLAC and IN2P3, Lyon, France, Jan 2001

Les Cottrell. Page created: January 22, 2001. Last update 1/30/01

Central Computer Access | Computer Networking | Network Group | ICFA-NTF Monitoring

Introduction

On January 22, 2001, Gilles Farrache of IN2P3 computer center, in Lyon France sent email reporting asymmetry in the throughput performance in the 2 directions between SLAC and IN2P3.

We start to use the new link Friday January the 19. What we discover is that
we have an asymmetric throughput. We are experiencing :

	20 Mbps SLAC/LYON
	29 Mbps LYON/SLAC

This asymetrie was already found in May last year between SLAC and
CERN and was never solved. The trouble ticket was TTS#6229 and 
Mike Collins was taking care of it.  

I have made mesureament between SLAC and CERN. The common path is
then STARTAP,ESNET,SLAC and I get the same asymmetrie :

CERN/SLAC : from 40 Mbps to 46 Mbps
SLAC/CERN : from 23 Mbps to 29 Mbps

The utilization of the utilization of the 155Mbps link between CERN and KPN is shown below.
MRTG plot of CERN utilization on its 155Mbps
link to KPN Amsterdam

MRTG plot of CERN utilization on its 155Mbps
link to KPN Amsterdam

The utilization of the 155Mbps link between SLAC and ESnet is shown below.
Utilization of SLAC-ESnet link

The spikes due to Gilles' tests are clearly visible in both utilization plots.

An earlier (December 2000) study of the connectivity between these sites is available at Network connectivity between SLAC and IN2P3, Lyon, France.

Loss

We ran a large (10000) number of 64 byte packet pings separated by 1 second intervals between ~11am and 2pm Monday 1/22/01, from SLAC to IN2P3 to ascertain the rough packet loss level to the 1 in 10000 level. There was no packet loss. We also ran sting many times, each time with a count of 2000 packets to see if we could ascertain one way packet losses. The results for roughly 35000 samples in each direction indicated that there was one packet lost in the CERN-SLAC direction and none in the SLAC-CERN direaction.

We looked at the Surveyor's one way loss and delay measurements between SLAC and CERN. These measurements are made roughly once/second/direction so in a complete day there would be about 86000 measurements. The measurements for January 14th, 15th, 16th, 2001 (the most recent available) showed no losses in either direction apart from a brief interruption on the 15th of a bout 15 minutes when there was 100% packet loss from CERN to SLAC. Measurements are not made when the GPS clocks are not available for synchronizing the measurements.

Routes

The traceroutes between SLAC and IN2P3 show that the routes are symmetric and short (8 hops) with most of the paths being within ESnet. The RTT in both directions is just under 150msecs.

The pingroutes between the two sites indicate the losses are less than 1% (i.e. no losses in 100 pings) (the apparent loss from IN2P3 to RTR-MSFC-DMZ.slac.stanford.edu is due to the way the router is configured).

The pchar path characteristics measured from SLAC to IN2P3 indicate that the bottleneck bandwidth to the IN2P3 site is about 30Mbps and is between the chi-snv.es.net and 192.70.69.14 192.70.69.14 is a PHYnet (as789) router interface of a p2p peering at AADS with ESnet. PHYnet pcr/scr is set to 40Mbps/30Mbps where CERN's pcr/scr is 149m/149m. This bottleneck limit is roughly confirmed by the bulk throughput measurements between SLAC and IN2P3 made by Gilles. We also used pipechar to charaerize the path. The pipechar results also show an ~30Mbps bottleneck close to ccasn01.in2p3.fr. The variability of the pipechar results (measured minutes apart) make one suspicious of how much one can read into them.

Historical performance

Using the IEPM/PingER data we can investigate the long term performance, in particular the packet loss, between IN2P3 and various PinGER monitoring sites around the world. The graph below shows the RTT and loss between SLAC and IN2P3 from December 2000 through January 2001. the reduction in RTT from about 170msec to 150msec around January 19th, 2001 is clearly visible. This is probably correlated in the link upgrade referred to by Gilles in his email above. the variability of the RTT indicates that at some times the link is saturated to the length that queuing is observed.

Resolution

The following email from Gilles Farrache on 1/30/01 indicates what was done to solve the problem:

The problem have been solved after making change on the definition on the PVC
between IN2P3 and ESnet (migrating the Renater part to ubr+ and changing
the vbr caracteristic of the ESnet side). We have now the max throughput of

	32 Mbps ATM SLAC/IN2P3
	30 Mbps ATM IN2P3/SLAC

As the router on the ESnet side is not able to run ubr+ we now have stop our
changes.

Page owner: Les Cottrell