In Panels, Context is lingo for a wrapper around any significant object. By default, Panels supports 'Node', 'Taxonomy term' (and terms as mentioned above), 'Taxonomy vocabulary', and 'User' as contexts.
These contexts can get into a display in more than one way; right now, there are three ways that a panel page can acquire a context:
1) Via arguments. Under the 'advanced' tab in panel pages, there is a list of possible arguments. Once added to the page, the argument is converted to a context, and this context becomes available for panes to use. Also, arguments can be selective; the node ID argument can restrict itself to a node type; if the node ID given is not a node of one of these types, no context will be created. (And the argument can tell the system to 404 or just let it pass and the panel will run without that context).
2) Simply added. Under the 'advanced' tab (currently in -dev only and will be in Alpha14), there is a group a Context section. Clicking 'add' on one will bring up a dialog, where a specific object is selected. For example, when adding the Node context, the popup will provide the autocomplete to select a node by title or nid.
3) By relationship. Once a context is in a page via any method, more contexts can be added by relationship. For example, if I have a 'node', I can extract one or more 'term' contexts from it by the obvious relationship. As well, I can come up with another node via 'book parent'. I can come up with a user reference from a node too, by getting the author. Once CCK integration is in place (currently it is not) I could come up with nodes via node references and users via user reference.
Once you have contexts in the system, this allows panel panes to appear that require contexts. For example, there is a 'node content' pane that simply renders a node. Without a node, it can't do anything; this context is required. If a node context isn't in the system, you won't even see this as an option. But once the display has a node context available, Panels will allow you to add the node context pane. Even better, if you happen to have multiple node contexts, in the popup to add the pane, you can choose which node context to render.
Panels will also let you convert the contexts into Views arguments, so that you can more visually control how Views get their data; you will no longer need to use the %1 and @1 methods of giving information to Views.
Here's a random example of what can be done
I can create a panel page which accepts a nid as an argument; that argument creates a node context. I then create a relationship which relates that node to the 'top level book', which means it will follow the book chain all the way up. I also create a relationship to the node's term in a specific vocabulary. On the display, I put the node content; to the side, I put the node content for the book parent, which is a description of the book. I also put the book navigation for the book parent, so that always is available. I also put the term description, so that it describes how this node is categorized. And I can also put a view of the term, so I can get links to the 5 most important nodes for that term.
Contexts, arguments, relationships and content types are all defined via plugins, which means they're in individual .inc files. But all plugins are also defined through module hooks. It is easy for a module to make additional contexts, arguments, etc available. In fact, specializing a 'node' argument so that it only accepts nodes with a certain criteria is something that makes a lot of sense.