End-to-end Arguments in System Design April 2, 2009

In this paper, the authors argue that no amount of cleverness, redundancy, error detection, etc. at a low level of the system can obviate the need to provide end-to-end protections at the application level. The crux of their argument is that the potential sources of data corruption and loss include sources which are fundamentally uncorrectable at the low level. As an example, no amount of network protections can prevent an intermediate host from corrupting data as it passes it from network link to network link. In addition to being insufficient to prevent all errors, low-level protections can actually add overhead which can cause unnecessary performance degradation. In the best case, the low-level functions can serve as a performance optimization to reduce the amount of time before an error is detected, but cannot provide all of the protection necessary.

In my opinion, the biggest problem with this paper is that it ignores the programming overhead imposed at the application level as a result of having to implement everything in an end-to-end manner. While it is true that the application level will always have to provide some kind of error checking and data verification, there is no need to make every program perform some of the tasks which can be performed at a low level. In addition to the programming overhead, there is also a potential performance overhead. If functions are implemented at the hardware level, it is possible to design specialized hardware which can provide the functionality much faster than equivalent application-level software could.

Future research in this field needs to be domain-specific, since the mechanisms for packet-switched networks likely overlap little with those for copying data between two hard disks in a system. As mentioned in the paper, the key for determinining the system’s position on the low-level/end-to-end spectrum is a careful analysis of which functions can be more efficiently and effectively implemented at a low level.

Fundamentally, the concept of this paper is sound. There are virtually no applications in which there does not need to be some end-to-end checking regardless of the underlying implementation. The key is striking a compromise between efficiency, modularity, and effectiveness.

End-to-end Arguments in System Design April 2, 2009

(i) the three most important things the paper says

The main important aspect of the paper is describing the end-to-end argument as  a design principle. Placing functionality at the lower layers of abstraction may be redundant, incomplete, of little value, or increase cost, compared to providing the functionality at a higher layer or level of abstraction.

Then the paper goes into detail about specific examples such as the file transfer application and makes the point that “it would be too simplistic to conclude that the lower levels should play no part in obtaining reliability”. This leads to talking about performance issues with implementing both functionality at a lower level and a higher level and “using performance to justify placing functions in a low-level subsystem must be done carefully”. It must be designed carefully because “the same or better performance can be achieved at a higher level” due to reduced overhead and application specific knowledge and increased knowledge that the lower layers will not have. But also in some cases, having that functionality at a lower layer will be more efficient if there is low overhead and it is a common subsystem to many applications.

Finally, the paper makes the conclusion that “Thus the end-to-end argument is not an absolute rule, but rather a guideline that helps in application and protocol design analysis; one must use some care to identify the end points to which the argument should be applied.”

(ii) the most glaring problem with the paper

The paper acknowledges that there is still a need for both end-to-end and low level designs depending on the application and future needs. A glaring problem is identifying the end in end-to-end design. This can be specific to the application, properties of the application, any number of other properties and variables. One interesting example the paper mentions is RISC processor architecture as an example of end-to-end system design.

(iii) the future research directions of the work.

There are many future research directions taking this end-to-end argument and applying it to design of operating systems, cryptographic systems, distributed systems, next generation network protocols, real life situations, etc…