Read an Excerpt
Professional XMPP Programming with JavaScript and jQuery
By Jack Moffitt John Wiley & Sons
Copyright © 2010 John Wiley & Sons, Ltd
All right reserved. ISBN: 978-0-470-54071-8
Chapter One
Getting to Know XMPP
WHAT'S IN THIS CHAPTER?
* The history of XMPP * XMPP networks and connections * XMPP's three building block stanzas
The eXtensible Messaging and Presence Protocol (XMPP) is, at its most basic level, a protocol for moving small, structured pieces of data between two places. From this humble basis, it has been used to build large-scale instant messaging systems, Internet gaming platforms, search engines, collaboration spaces, and voice and video conferencing systems. More unique applications appear every day, further demonstrating how versatile and powerful XMPP can be.
XMPP is made of a few small building blocks, and on top of these primitives many larger constructions have been made. Within XMPP are systems for building publish-subscribe services, multi-user chat, form retrieval and processing, service discovery, real-time data transfer, privacy control, and remote procedure calls. Often, XMPP programmers create their own, unique constructions that are fitted exactly for the problem at hand.
Most social media constructs that have propelled web sites like Facebook, MySpace, and Twitter into the forefront are also baked into XMPP. Within XMPP, you'll find rosters full of contacts that create a social graph with directed or undirected edges. Presence notifications are sent automatically when contacts come online and go offline, and private and public messages are the bread and butter application of XMPP systems. Developers will sometimes choose XMPP as the underlying technology layer simply because it gives them many social features for free, leaving them to concentrate on the unique pieces of their application.
The possibilities are vast, but before you can begin, you need to know about XMPP's different pieces and how they fit together into a cohesive whole.
WHAT IS XMPP?
XMPP, like all protocols, defines a format for moving data between two or more communicating entities. In XMPP's case, the entities are normally a client and a server, although it also allows for peer-to-peer communication between two servers or two clients. Many XMPP servers exist on the Internet, accessible to all, and form a federated network of interconnected systems.
Data exchanged over XMPP is in XML, giving the communication a rich, extensible structure. Many modern protocols forgo the bandwidth savings of a binary encoding for the more practical feature of being human readable and therefore easily debugged. XMPP's choice to piggyback on XML means that it can take advantage of the large amount of knowledge and supporting software for dealing with XML.
One major feature XMPP gets by using XML is XML's extensibility. It is extremely easy to add new features to the protocol that are both backward and forward compatible. This extensibility is put to great use in the more than 200 protocol extensions registered with the XMPP Standards Foundation and has provided developers with a rich and practically unlimited set of tools.
XML is known primarily as a document format, but in XMPP, XML data is organized as a pair of streams, one stream for each direction of communication. Each XML stream consists of an opening element, followed by XMPP stanzas and other top-level elements, and then a closing element. Each XMPP stanza is a first-level child element of the stream with all its descendent elements and attributes. At the end of an XMPP connection, the two streams form a pair of valid XML documents.
XMPP stanzas make up the core part of the protocol, and XMPP applications are concerned with sending and responding to various kinds of stanzas. Stanzas may contain information about other entities' availability on the network, personal messages similar to e-mail, or structured communication intended for computer processing. An example stanza is shown here:
What think you of books?
In a typical client-server XMPP session, a stanza such as this one from Elizabeth to Mr. Darcy will travel from Elizabeth's client to her server. Her server will notice that it is addressed to an entity on a remote server and will establish an XMPP connection with the remote server and forward the message there. This communication between servers resembles the e-mail network, but unlike e-mail servers, XMPP servers always communicate directly with each other and not through intermediate servers.
This direct communication eliminates some common vectors for spam and unauthorized messages. This is just one of the many ways in which XMPP is designed for security. It also supports encrypted communications between endpoints through use of Transport Layer Security (TLS) and strong authentication mechanisms via Simple Authentication and Security Layers (SASL).
XMPP is designed for the exchange of small bits of information, not large blobs of binary data. XMPP can, however, be used to negotiate and set up out-of-band or in-band transports, which can move large blocks from point to point. For these kinds of transfers, XMPP functions as a signaling layer.
The focus on small, structured bits of data gives the XMPP protocol extremely low latency and makes it extremely useful for real-time applications. These applications, which include collaborative spaces, games, and synchronization, are driving XMPP's growth in popularity as developers experiment with the real-time Web.
You will see how easy it is to make real-time web applications through this book's examples. By the end of the book you should have a thorough understanding of why so many people are excited about XMPP's power and promise.
A BRIEF HISTORY OF XMPP
The XMPP protocol is now more than 10 years old, and it has come a long way from its humble beginnings. Much of XMPP's design is due to the environment in which XMPP was created, and the history of XMPP provides an interesting case study in how open protocols foster adoption and innovation.
In 1996, Mirabilis released ICQ, which popularized rapid, personal communication among Internet users. Its use spread rapidly, and before long other companies were releasing similar products. In 1997, AOL launched AOL Instant Messenger. Yahoo followed suit in 1998 with Yahoo Pager (eventually renamed Yahoo Messenger), and in 1999 Microsoft finally joined the competition with MSN Messenger (now Windows Live Messenger).
Each of these instant messaging applications was tied to a proprietary protocol and network run by the companies that made them. Users of ICQ could not talk to Yahoo users and vice versa. It became common for users to run more than one of these applications to be able to talk to all of their contacts because no single vendor claimed 100% market share.
It didn't take long before developers desired to write their own clients for these proprietary IM networks. Some wished to make multiprotocol clients that could unite two or more of the IM networks, and others wanted to bring these applications to operating systems other than Microsoft Windows and Apple's Mac OS. These developers ran into many roadblocks; they had to reverse-engineer undocumented protocols, and the IM networks aggressively changed the protocol to thwart third-party developers.
It was in this climate that the idea for an open, decentralized IM network and protocol was born.
Jeremie Miller announced the Jabber project in January of 1999. Jabber was a decentralized instant messaging protocol based on XML and a server implementation called jabberd. A community immediately formed around the protocol and implementations spawning more clients and more ideas. By May of 2000, the core protocols were stabilized and jabberd reached a production release.
The Jabber Software Foundation (JSF) was founded in 2001 to coordinate the efforts around the Jabber protocol and its implementations. By late 2002, the JSF had submitted the core protocol specifications to the IETF process, and an IETF working group was formed. In October 2004, this standards process produced improved versions of the Jabber protocols, renamed XMPP, documented as RFCs 3920, 3921, 3922, and 3923.
During the protocol's early life, developers continued to expand its possibilities by submitting protocol extensions to the JSF. These extensions were called Jabber Extension Proposals (JEPs).
Eventually the JSF and the extensions followed the naming change from Jabber to XMPP and became the XMPP Standards Foundation (XSF) and XMPP Extension Proposals (XEPs).
By 2005, large-scale deployments of XMPP technology were well underway, highlighted by the launch of Google Talk, Google's own XMPP-based IM service.
Today, the XMPP ecosystem is quite large. Nearly 300 extensions have been accepted as XEPs, and dozens of client and server implementations have been created - both commercial and open source. Software developers of virtually any programming language can find a library to speed their XMPP application development efforts.
XMPP applications started out very IM-centric, reflecting its origins, but developers have found XMPP to be quite capable for a number of applications that weren't originally foreseen including search engines and synchronization software. This utility is a testament to the power of an open system and open standardization process.
Most recently, the IETF has formed a new XMPP working group to prepare the next versions of the XMPP specifications, incorporating all the knowledge gained since the original RFCs were published. XMPP continues to be refined and extended so that application developers and Internet users will always have an open, decentralized communications protocol.
THE XMPP NETWORK
Any XMPP network is composed of a number of actors. These actors can be categorized as servers, clients, components, and server plug-ins. An XMPP developer will write code to create or modify one of these types of actors. Each actor has its place on the XMPP network's stage.
Servers
XMPP servers, or more accurately, XMPP entities speaking the server-to-server protocol or the server end of the client-to-server protocol, are the circulatory system of any XMPP network. A server's job is to route stanzas, whether they are internal from one user to another or from a local user to a user on a remote server.
The set of XMPP servers that can mutually communicate forms an XMPP network. The set of public XMPP servers forms the global, federated XMPP network. If a server does not speak the server-to-server protocol, it becomes an island, unable to communicate with external servers.
An XMPP server will usually allow users to connect to it. It is, however, also possible to write applications or services that speak the server-to-server protocol directly in order to improve efficiency by eliminating routing overhead.
Anyone can run an XMPP server, and full-featured servers are available for nearly every platform. Ejabberd, Openfire, and Tigase are three popular open source choices that will work on Windows, Mac OS X, or Linux systems. Several commercial XMPP servers are available as well, including M-Link and Jabber XCP.
Clients
The majority of XMPP entities are clients, which connect to XMPP servers via the client-to-server protocol. Many of these entities are human-driven, traditional IM users, but there are also automated services running as bots.
Clients must authenticate to an XMPP server somewhere. The server routes all stanzas the client sends to the appropriate destination. The server also manages several aspects of the clients' sessions, including their roster and their bare address, which you see more of shortly.
All of the applications in this book are written as client applications. This is typically the starting point of most XMPP development. For applications without a user focus or with demanding needs, it is often preferable to create a different kind of entity, such as a server component.
Components
Clients are not the only things that may connect to XMPP servers; most servers also support external server components. These components augment the behavior of the server by adding some new service. These components have their own identity and address within the server, but run externally and communicate over a component protocol.
The component protocol (defined in XEP-0114) enables developers to create server extensions in a server-agnostic way. Any component using the protocol can run on any server that speaks the component protocol (assuming it doesn't use some special feature specific to a particular server). A multi-user chat service is a typical example of something that is often implemented as a component.
Components also authenticate to the server, but this authentication is simpler than the full SASL authentication for clients. Typically authentication is done with a simple password.
Each component becomes a separately addressable entity within the server and appears to the outside world as a sub-server. XMPP servers do not manage anything beyond basic stanza routing on behalf of connected components. This allows great freedom to component developers to do things exactly as they want, but places greater responsibility on them when they need functionality such as rosters and presence management.
The server also allows a component to internally route or manage stanzas for itself. A component can therefore create separately addressable pieces to be used as rooms, users, or whatever the developer requires. This is something that a client session cannot do and can be used to create really elegant services.
Finally, because components do not have resources managed for them, services that operate with many users or with a high amount of traffic can manage their own resources in a way that makes sense for their purpose. Developers often create services as client bots, only to discover later that the server's roster management capabilities often do not scale well to thousands upon thousands of contacts. Components can manage rosters, if they have them at all, in whichever way makes sense for the task and scale required.
Plug-ins
Many XMPP servers can also be extended via plug-ins. These plug-ins are usually written in the same programming language as the server itself and run inside the server's processes. Their purpose overlaps to a large degree with external components, but plug-ins may also access internal server data structures and change core server behavior.
The virtually limitless abilities afforded to server plug-ins come with a cost; plug-ins are not portable between different servers. A different server may be written in a completely different language, and its internal data structures may differ radically. This cost aside, plug-ins are sometimes the only way to get a particular job done.
Plug-ins have reduced overhead compared to components because they do not need to communicate over a network socket. They also need not parse or serialize XML and can, instead, work directly with internal server representations of stanzas. This can lead to much needed performance improvements when the application must scale.
XMPP ADDRESSING
Every entity on an XMPP network will have one or more addresses, or JIDs. JIDs (short for jabber identifiers) can take a variety of forms, but they normally look just like e-mail addresses. darcy@ pemberley.lit and elizabeth@longbourn.lit are two examples of JIDs.
Each JID is made up of up to three pieces, the local part, the domain, and the resource. The domain portion is always required, but the other two pieces are optional, depending on their context.
The domain is the resolvable DNS name of the entity - a server, component, or plug-in. A JID consisting of just a domain is valid and addresses a server. Stanzas addressed to a domain are handled by the server itself and potentially routed to a component or plug-in.
The local part usually identifies a particular user at a domain. It appears at the beginning of a JID, before the domain, and it is separated from the rest of the JID by the @ character, just like the local part of an e-mail address. The local part can also be used to identify other objects; a multi-user chat service will expose each room as a JID where the local part references the room.
(Continues...)
Excerpted from Professional XMPP Programming with JavaScript and jQuery by Jack Moffitt Copyright © 2010 by John Wiley & Sons, Ltd. Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
