Our Virtual Disk service is provided by the Virtual Disk Server. The approach is similar to the idea of virtual memory in an operating system. Clients may write to any location in that huge memory, but only a small subset of that memory actually represents real data. Providing the Virtual Disk as a service accessed via the network allows us to support concurrent access more easily than with a straight file-based implementation. The Virtual Disk Server allows the disk to addressed in pages, each containing 4K of data. Since none of these pages overlap, read and write requests are idempotent. This allows the Server to serialize the incoming requests in any order it likes. Write requests for the same page that have arrived out of order need to handled specially though.
The Virtual Disk Server uses the local filesystem for storage. Each page to be stored is recorded in an Indirection Map file which stores the page address and the index of the corresponding data block. The data blocks reside in another file. The Indirection Map is kept sorted by address, page lookups are done using a binary search.
Clients interact with the Virtual Disk Server by sending it read or write requests. Each request identifies the address of page to be read or written and the IP address of the client. When submitting requests it is the responsibility of the client to resend a request in the event of no reply from the Server (We use a connectionless protocol for communication). This reduces the load on the server and gives it the freedom to discard requests when under heavy load.
