Directory Services: DNS and LDAP
While attending the caBIG Architecture Workspace face-to-face in Chicago last month, I realized during a discussion that I probably didn’t know enough about directory services. One of the major challenges faced in designing the caBIG architecture is the matter of serving up Common Data Elements, or CDEs. These are standardized vocabulary terms used to compose a structured informational representation of a clinical outcome, expression level measurements, etc. Because they will be used throughout caBIG, some service must be provided to allow components of caBIG to lookup these vocabulary terms, e.g. to fill in the details of a particular dataset. This vocabulary service must be distributed and highly available for even a moderately mature grid to function properly, and avoid CDE lookups becoming a grid bottleneck.
At the face-to-face, Frank Hartel asked the room what sorts of servers and server architectures might be needed to fill such a requirement. I chimed in, naively, that we might learn something from the DNS, because of its ubiquity and high availability. Frank responded that he/they were thinking more of something like LDAP. Now, I’d heard of LDAP and knew it had something to do with looking up organizational directory information (names, email addresses, phone numbers), but had no real hold on the scope of the LDAP standard and the problems it was capable of addressing.
Since then, I’ve tried to do my homework, have read both about the DNS and LDAP directory services and have thought about how they might be applied to serve vocabularies to the caBIG grid. The following are some preliminary thoughts.
Both DNS and LDAP serve information organized in a tree of nodes, each node having a particular class and containing some number of attributes. In DNS, this tree is called the Domain Name Space, and in LDAP it is called the Directory Information Tree (DIT). Each node of the tree also has a unique name. In DNS this name is the fully-qualified domain name (FQDN), and in LDAP this is the distinguished name (DN). Highly distinguished indeed. In DNS there are not that many node classes, by far the most common being the IN, or Internet, class. By contrast, there are a very large number of LDAP node objectclasses. Each LDAP objectclass must be assigned a globally unique object identifier (OID), which look like 1.1.3.232.23.242.1.52, i.e. a string of integers separated by periods. The OIDs fit into their own tree according to a hierarchical structure imposed on the organizations to which they are assigned (or something like that).
The attributes of a node in DNS are called resource records (RRs). The most well-known type of resource record is the A, or address, record, which for IN nodes stores an IP address associated with the FQDN of the node. Common LDAP attributes are, for example, dc (domain component), dn (distinguished name, which, as an attribute, is a relative name, or the most specific part of the full DN of the node), o (organization), cn (common name), and sn (surname).
In DNS, a node’s class does not determine which attributes it may contain. Any node can potentially have any attribute. The class refers to type of network about which information is provided (IN = internet, CH = chaos). Therefore, extending DNS is a matter of adding new attributes, or types of resource records. By contrast, in LDAP, a node’s objectclass determines the types of attributes that node must and may contain. Extending LDAP is more flexible than extending DNS, and would require writing new objectclass and attribute definitions in an LDAP schema.
Both LDAP and DNS provide for information about different parts of the tree being served by different servers on the network. The directory tree is split into “zones” (the DNS word) or “partitions” (the LDAP word), which are subgraphs of elements that share a common ancestor plus the common ancestor (the “root”). A particular server can refer requests that it receives for parts of the directory tree outside of its zone/paritition to other servers. Servers can also cache results of requests referred to other servers, as well as cache results of requests performed on their local directory. Finally, a given server can be made redundant for load-balancing and failover purposes. Synchronization between the master server and its replicas occurs via a pull model in DNS (replicas periodically poll the master for updates), whereas in LDAP either a push (the master notifies the replicas of updates) or a pull model can be used.
On the face of things, the two technologies seem rather equivalent. However, DNS is decidedly the more mature technology. It servers millions of users every day, there are thousands of DNS servers deployed on the network, and every computer has a command-line tool to debug DNS queries (dig or nslookup). LDAP is a standard for organization-wide directory services, but it is not in great use cross-organizationally (as far as I know). DNS was designed to provide a fairly narrow range of directory services; by contrast, LDAP is much more general.
Thus far, the two tools have largely been applied to very different problems, so it’s difficult to compare how the two might fare at the same task.