Projects/Usability/NWI - KDE TechBase

The idea of NWI (Nested Windows Interface) emerged from the discussion about MDI (Multiple Document Interface) on kde-usability (and later kde-usability-devel) mailing list — so far each application had to implement the most suitable UI by its own (TDI in Konqueror, SDI in Okular) which led to inconsistent behaviour ("is closing the last document possible?").

Kpdf + manual "grouping" of KWrite and Konqueror

NWI is thought as a uniform UI on one hand, while helping to increase productivity by extending the idea of tiles and tabs on the other. Consider an example: you edit some data in one application (e.g. an html file in KWrite), and you view the results using another application (in this instance, Konqueror). So although there is no relation between KWrite and Konqueror, for this task — a task you perform right now — those two applications are related. Currently — if you run more than those two applications — when you switch to KWrite, the Window Manager (WM) will bring only KWrite to the front. If another window has meanwhile covered Konqueror, you now have to bring that window to the front by yourself. This is tiresome. With grouping, KWrite and Konqueror would act as one — you switch to container, all applications are put in front.

NWI moves document grouping from the application level to the Window Manager level. Multiple applications — even those using unrelated toolkits (e.g. KDE and Gnome) — can be grouped and managed as a single entity. You are probably familiar with tabs from modern web browsers, and split views from e.g. Kate. Now imagine every program you use having tab support, split view support, and being able to combine them in any combination..

What is it?

Probably you are already familiar with several User Interface designs; TDI, MDI, SDI, and so on. Some of them have proven to be very useful, some of them, not especially. In the NWI world, the most basic — yet the more flexible — UI you can think of is built from those below:

Single Application Interface (SAI)

This is really the simplest UI; no magic here. All you have is just an window of an application instance with some data — if a "window" can hold only one document, it is SAI. Examples: Kpdf, Okular.

Applications can can postpone "real" UI mode — "decide later whether this SAI or TAI". In other words, there is an option "show tabs" (per each application supporting such mode), which translates to:
off → start as SAI, user can add another tab later
on → from the start work as TAI (show even single tab)

In another sense, SAI can be thought of as a leaf; an actual window, rather than one of the container types that follow.

Grid Applications Interface (GAI)

A more accurate name would be Tiling Applications Interface, but this would produce the same acronym as Tabbed Applications Interface.

GAI is a container that displays multiple "views" (a window, document, or other collection of data or controls) next to each other, vertically and/or horizontally, usually with no space between. From a geometrical point of view, GAI container behaves as one window with many panes. Example: Kate, KMail.

GAI has one shared titlebar (for the container), plus at user request each embedded window may have a mini-titlebar.

When you finish grouping, you still can change the proportions of the grid "panes". You can also rearrange them — via context menu or simply like dragging docked windows. Changing the size of one pane will generally change the size of adjacent panes, so that the entire container is filled.

Floating Application Interface (FAI)

The most common FAI container is desktop; windows in an FAI container can be moved freely, and altering the geometry of one window does not have an effect on any others. While the desktop will default to FAI mode, it is possible to switch it to another type. When using multiple desktops, each desktop is considered a separate container; as such, individual desktops may be set to different container types (e.g. desktop 1 is TAI, desktop 2 is GAI, desktops 3 and 4 are FAI).

The FAI container can be any non-root container as well — being able to convert to/from any container, FAI helps reorganizing windows inside GAI. Non-root FAI is what some people (especially those familiar with its use in the Microsoft world) call "MDI".

A minimized window in an FAI root container (desktop) is hidden and accessible from the task bar/panel. Windows in a non-root FAI container "minimize" to just a title bar (much like shading), so they are directly accessible all the time.

When resizing the parent container, the visible portions of the embedded windows are preserved (the embedded windows are "pushed"). If this is no longer possible, the windows are resized.

Default options:
[ ] Show scrollbars (when needed) and allow windows to be moved outside the visible area.
(*) Use mini-task-bar
( ) Minimize windows to bottom of window (appear as shaded title bars)
( ) Minimize windows in place (minimize == shade)

Writing the FAI backend to allow scroll bars, and rewriting the root as another FAI container, should have the side effect of giving us the much-requested "virtual desktop" feature, or at least putting us very close to having it.

Tabbed Applications Interface (TAI)

If you've used a modern web browser, you have almost certainly experienced TAI in the form of TDI (Tabbed Document Interface). You launch one application instance, and you can then open multiple data views (documents, web pages, even shell sessions) in one window, with one per tab. Example: Konqueror, Konsole.

TAI goes a little further than TDI, because you can arrange several application instances in tabs and put them into one window. So in one tab you can have Konqueror displaying web page, and in the second tab you can have Okular displaying pdf file. (Microsoft Internet Explorer does this to a limited extent with Adobe Acrobat integration. Imagine being able to open any document from a web page in an appropriate viewer and have it be just another tab.)

By default, tabs are shown at the top of the window, but the user can choose for them to be on any side. This can be configured both as a global default, and per container.

Nested Windows Interface (NWI)

Once the container is arranged, you can think of it as a window. As such, you can put into another container, thus nesting containers. Therefore, the UI in general is called NWI; Nested Windows Interface.

The visual aspect of containers is controlled by the window decorator, so different decorators may make different choices regarding how containers are displayed (for example, TAI might have a title bar and tab strip, or might combine the two in the manner of Google's Chrome browser).

The desktop and each container type have their own default setting whether they should display a unified (shared) menu, or allow menus to be displayed in the child windows as they normally would. Individual containers can also be configured for regular/shared or use the container-type default.

When displaying a shared menu, the menu for the active leaf (that is, the descendent window that has input focus) is displayed in the top of the container, and no other menus are displayed in that container (not limited to immediate children) — "take all" mechanism.

Note that typical "hide menubar" option affects only application menu, not shared menu. To hide shared menubar user has to select "hide container menubar" option.

{ MAYBE: in case with shared menu there could be added such container entries like "close window", "create preset", "convert to", "help", so the user would not have to use context menu to pick up those. And such entries would show up even if app does not have menu on its own. }

Default options:
[ ] GAI uses shared menu
[ ] FAI uses shared menu
[ ] TAI uses shared menu
[ ] override the container setting for the desktop
[ ] desktop uses shared menu

Features

Containers cannot get focus — only the windows of application instances within can get focus — however they appear in window lists (e.g. window switcher, task bar) as if they could get the focus (you can think of the container as focus-proxy). Requesting that a container be given focus (e.g. by window switcher, task bar, or clicking the container's title bar) will give focus to the window within that container that most recently had focus. If that window is itself a container, then the window in that container that most recently had focus will receive it, and so on until a leaf (a non-container window) is reached.

Session management

When you turn the session management on (at KDE level), SM not only restores the application instances, but also recreates containers.

It is also possible to save the current application instance set of the container as a preset (e.g. KWrite and Konqueror in GAI), and then later launch such group at once. This saves recreating commonly-used groups.

Data saved:

set of windows of the container
container type
container class name
container title
container shared menu settings

At some point, we would like to be able to configure launching saved groups with arguments (e.g. file names), including which new application instances should get what arguments. This would allow e.g. an action on HTML documents to open the aforementioned KWrite+Konqueror group for simultaneous viewing and editing of the document(s).

Container presentation

Containers will use a window title that may represent information about the active window and/or the container type. This can be changed by the user, per container, in a similar manner to naming tabs in Konsole. Special character sequences may be used to substitute information such as the window title of the active child, or the descendent which has input focus (or "would have", if the container is given focus). These titles are used in the container's title bar, where applicable, and in task lists (task bar, window switcher, etc.).

The task bar shows (directly) only those windows that live directly in the root container. When user clicks LMB on container entry in the task bar, the list of all embedded windows is shown (containers show items which expand to the windows they contain, and so on). As with collapsed tasks, containers show a small number with the number of windows they contain.

NWI also provides options for controlling the way the task bar manages collapsing (merging of task entries into a single top-level entry):
[x] collapse similar windows on the task bar
[ ] collapse similar windows in pop-up lists

A pop-up list would be e.g. the menu that appears when selecting a task bar entry for a container or collapsed entry.

{ nicer screenshot to show both of them in action }

Similar windows are those belonging to the same application, or containers with the same class. By default, each container is assigned a "class name" from its initial: contents and type. The class name can also be changed by the user for fine control over collapsing.

Managing the containers

Except for the root container (the desktop), empty containers are not allowed. Thus you have to point out which windows you would like to group or convert one of the windows to be container and add another window to it. Once a container is created, the user can label it as she/he likes (the label will be used as the container title, e.g. in the titlebar, task bar, switcher, etc.).

Arranging windows as container

launch applications as you like
RMB on the desktop to get context menu
choose "create container" (control dialog appears with "accept" and "cancel" buttons and container type selection)
select desired windows
click LMB on "accept" or press [Enter] key — container is created

To select windows:

with Shift+LMB pressed down select region over the windows (like in KSnapshot)
with Ctrl pressed down click LMB on the window (to deselect click again)

You can also run any switcher when selecting; this is useful for FAI containers.

Incremental adding

You can:

add a new application instance of the same type as the current window, as a new sibling of the current window (equivalent to the "new tab" feature of many web browsers)
add a duplicate of the current application instance (i.e. it "copies" the application instance and the document)
add the "hollow" application — after that you drag&drop the desired window, or start a new application instance; it will be "sucked" into the container
drag&drop a window directly to the container without creating the hollow application

There can be only one hollow application instance for the entire session (all desktops). If the user tries to create a new hollow application instance, the previous one disappears. Existence of the hollow application instance indicates that the next user action will be adding a window to that container (thus drag&drop is easier in such case). Note that dialogs are not "sucked in" (this would be inconvenient if the dialog is the application launcher!).

TAI has special options for incremental adding:
( ) add new entry next to the current one
(*) add new entry as the last one

Disassembling the container

user can convert any container to any other type (for example FAI to GAI)
user can tear off any window from the container to other container by drag&drop, even if the source and target containers are different types

From the accepting container's point of view, tearing off is the same as adding a window.

Please note that moving embedded windows within a container and tearing them off are distinct actions. The WM provides an option to control that behavior:
[x] allow tearing off window while rearranging

If this option is turned off, then a window cannot "fall off" the container while being moved.

Closing documents and windows

Empty containers (except for desktop) are not allowed; empty application windows (that is, application windows with no document opened) are allowed.

close current document
quit current application
close current application window/tab
close current application window/tab siblings (i.e. close all children of the parent, except the current one)
close parent container of the current application window/tab (not counting desktop)
close topmost container (not counting desktop)

Closing an entire container means that all windows within the container are closed. Most applications quit when their windows are closed. Some kinds of application may continue to run after their windows are closed (systray applications, in particular, whose systray icons remain after their windows are closed). The use of containers does not change this behavior.

When the window which had focus is closed, focus is given to the next most recent window in the historic bookkeeping. A global option allows this behavior to be changed for TAI containers.

When a window in a tabbed container is closed:
(*) Give focus to the previously-active window
( ) Give focus to the spatially adjacent window
[ ] Give focus to nearest sibling of the same application kind, when possible

The second option prefers the next window, but uses the previous window if the window closed was last in the spatial order.

Close current document

Close current document is only partly in the scope of NWI, as it is implemented at the application level. Applications may:

not implement close-document
always close the document, but never close the window
always close the window
conditionally close the window

This last case involves NWI communication, likely by sending a 'close-window-conditional' signal to the WM. The window will be closed if the window has at least one sibling (application instance) of the same application kind. When making this decision, "application instances" need not belong to the same process, although they do need to be running on the same machine. (See the glossary for a more precise definition of when application instances are considered as belonging to "the same application".)

The choice of behavior, and if any configuration is available, is left to the application.

Quit application

Quit application is not really in scope of NWI, but since it is related to closing application window(s), we list it for the purpose of reference and to show the difference between those two actions. This document does not address the question whether "quit application" should exit only the active window, the corresponding application process, or all instances of the application (which may belong to more than one process).

Container special menu items

Accept new application (add hollow application)
Convert to...
- GAI
- FAI
- TAI
Save set as...

Default key bindings

Closing
close current document	[ctrl+w]
quit current application instance	[ctrl+q]
close current application window/tab	<none> { alt+q? }
close current application window/tab siblings	<none>
close parent container of the current application window/tab	[alt+f4]
close topmost container	[alt+shift+f4]
Changing container type
to GAI	<none>
to FAI	<none>
to TAI	<none>
Adding applications
empty copy	[ctrl+n]
duplicate	[ctrl+d]
hollow	[ctrl+h]
start direct dragging	[ctrl+LMB]

Switchers

A switcher is a program that lets the user switch from application instance to application instance. Visual appearance is up to the switcher. The user will select one switcher to be the primary switcher; other switchers may be configured as "auxiliary switchers", with any number of auxiliary switchers possible at one time.

The primary switcher is activated by pressing and holding a "mod" key plus another key to make a navigation key. (For example, a navigation key might be [tab] or [shift-tab]; using mod as [alt] gives the switching keys commonly known today.) At that point, the primary switcher is active, and to keep it running, the user must continue to hold the mod key. Releasing the mod key will give focus to the window currently in the switcher; pressing the cancel key (default [escape]) will cancel the switch. Switchers may also (optionally) make use of the mouse.

Auxiliary switchers are triggered by a global key or key combination. The combination need not include a modifier, and it is not needed to hold a modifier to keep the switcher active. The switcher will be operational until the user commits the switch (default [enter]) or cancels the switch.

Mod is any modifier key allowed/recognized by X, except [ctrl] and [shift] — default mod key will likely be [alt] or [win].

When window is a container, after switching to it, focus is given as described at the top of the Features section. Each container remembers the last window to have focus.

Exchange

Switchers are used for switching (focus change) windows, but they have another purpose: exchanging windows. The user can exchange the marked window with any other one (not only siblings). After exchange, the marked window remains marked (exchange is not switch). Exchange means that windows take each other's "places" in a geometrical sense (which includes position in the container hierarchy).

Initially, the current application instance is marked.

Keyboard shortcut actions

(window, desktop) sequence switch
(window, desktop, global) historic switch
(window, desktop) spatial switch
(window) container-smart switch

The ones marked "global" switch through all desktops, marked "desktop" switch only on the current desktop, marked "window" are constrained to the parent container of the active window.

In TAI switches spatial up = spatial left and spatial down = spatial right (because of the tabs placement).

Scope of the switches:

sequence — direct children only
historic — direct children only
spatial — all children (i.e. children, grandchildren, and so on...)

Container smart switches:

GAI — historic
FAI — historic
TAI — spatial

User can choose (option) if she/he prefers switch-wrap on or off, for each switcher mode separately. For sequence and historic switching, wrap means after last comes the first (and vice versa); for spatial that after rightmost comes the leftmost, after lowest comes the highest (and vice versa).

Default settings:
[x] sequence wrap
[x] historic wrap
[ ] spatial wrap

Spatial and sequence switching properties

Both of these switching modes are based on window placement, so for TAI container they are equivalent.

The order of the sequence switching depends on KDE language settings (i.e. it is the same as reading "order"). Thus you can call sequence switching as switch-as-you-read (titlebars). It means that container edges are no limits for sequence switch, but they are limits for spatial switch.

Another observation — the window to the right of its neighbor is not always left for spatial switch; that is, spatial-switch-left followed by spatial-switch-right may not put you back where you started. For sequence switch there is such guarantee: when you switch next and then previous, you will always return to the same place.

Default keyboard bindings

How to read it — first goes the container. It means the switch is bounded to that type of container. The second goes action. For example "desktop up" means this actions go to next upper window in current desktop. For auxiliary switchers the default shortcuts are the same, just [mod] key is dropped out (i.e. [left] instead of [win+left]).

Sequence switch
window previous	[win+backspace]
window next	[win+shift+backspace]
desktop previous	[alt+backspace]
desktop next	[alt+shift+backspace]
Spatial switch
window up	[win+up]
window down	[win+down]
window right	[win+right]
window left	[win+left]
desktop up	[alt+up]
desktop down	[alt+down]
desktop right	[alt+right]
desktop left	[alt+left]
Historic switch
window back	[win+pgup]
window forward	[win+pgdn]
desktop back	[alt+pgup]
desktop forward	[alt+pgdn]
global back	[alt+ctrl+pgup]
global forward	[alt+ctrl+pgdn]
Smart switch
window previous	[win+shift+tab]
window next	[win+tab]
desktop previous	[alt+shift+tab]
desktop next	[alt+tab]
Other actions
cancel switch	[mod+escape]
commit switch	[mod+enter] (or just release [mod])
exchange	[mod+ins]

Primary switcher is activated by any of the above (except the last section) and stays active as long as mod is held or until user commits or cancels switch. Cancel does not undo exchanges made.

Glossary

application: An application kind in general sense (e.g. Konqueror). Also refers to the collection of all application instances which are of the same application kind running with the same access class (see below).
application instance: See application window. For this page, an instance is not the same as a process.
application window: A window of a particular application { usually used when discussing sibling windows where some may be the same application }. A process (as defined by the operating system) may have more than one application window.
container: A special type of window which is not an application instance, but contains one or more child windows.
leaf: An application instance or other window which is not a container; may also include dialogs, etc.
root: The node in the window tree corresponding to the desktop. The root has no parent, and its siblings are the other virtual desktops.
sibling: A window belonging to the same direct parent container as a reference window.
window: An object (area) in sense of the geometry which is either an application instance or a container.

When considering if two application instances belong to the same application, a number of factors which together comprise the "access class" (roughly referring to the expected permission set possessed by the application instance) are considered. These are the instance's security credentials { for now, only euid is considered } and host machine. Two windows are not considered as belonging to the same application if:

They are running from different machines (e.g. one local, one remote over X forwarding or specification of a remote X server)
They were started under different users
They do not belong to the same X session

It should be noted that the window manager only considers windows under its control — that is, those running in the same X session. An application instance in another session cannot be considered as another instance of an application, since the window manager would not know about such a window.