Projects/Plasma/ScreenManagement

Architecture

The management of multiple screens and hot-plug thereof will make extensive use of XRandR 1.2, there should be a fallback mechanism for systems not supporting it though. For the client (ie Plasma, but also others) this should be completely transparent, so the API needs to be designed to be able to give all information it can get, but also act sensible when there is little or no information about the screen configuration available.

The system should offer a set of configurations/profiles which will be autoselected depending on the pugged devices. Clients should be able to easily change the used configuration, which will then be remembered for the next time these devices are plugged in.

Configuration

The configuration for this might look like the following. The ideal solution would be to agree with the Gnome-folks on one config-spec. This is actually the xml as used by the current implementation (rev 856215).

<configuration name="single" modifiable="false" primary="0">
    <screen id="0">
    </screen>
</configuration>
<configuration name="presentation" modifiable="false" primary="0">
    <screen id="0">
    </screen>
    <screen id="1">
        <privacy>true</privacy>
        <right-of>0</right-of>
    </screen>
</configuration>
<configuration name="extended-right" modifiable="false" primary="0">
    <screen id="0">
    </screen>
    <screen id="1">
        <right-of>0</right-of>
    </screen>
</configuration>

<outputs configuration="single">
    <output name="TMDS-1">
    </output>
    <output name="LVDS" screen="0">
        <vendor>X</vendor>
        <product>0x1234</product>
        <serial>0x12345678</serial>
        <width>1280</width>
        <height>800</height>
        <refresh-rate>60</refresh-rate>
        <rotation>0</rotation>
        <reflect-x>no</reflect-x>
        <reflect-y>no</reflect-y>
    </output>
    <output name="VGA">
    </output>
</outputs>
<outputs configuration="presentation">
    <output name="TMDS-1">
    </output>
    <output name="LVDS" screen="0">
        <vendor>X</vendor>
        <product>0x1234</product>
        <serial>0x12345678</serial>
        <width>1280</width>
        <height>800</height>
        <refresh-rate>60</refresh-rate>
        <rotation>normal</rotation>
        <reflect-x>no</reflect-x>
        <reflect-y>no</reflect-y>
    </output>
    <output name="VGA" screen="1">
        <vendor>Y</vendor>
        <product>0x9876</product>
        <serial>0x98765432</serial>
        <width>1024</width>
        <height>768</height>
        <refresh-rate>60</refresh-rate>
        <rotation>normal</rotation>
        <reflect-x>no</reflect-x>
        <reflect-y>no</reflect-y>
    </output>
</outputs>
<outputs configuration="extended-right">
    <output name="TMDS-1" screen="0">
        <vendor>Z</vendor>
        <product>0xABCD</product>
        <serial>0xABCDEF01</serial>
        <width>1680</width>
        <height>1050</height>
        <refresh-rate>60</refresh-rate>
        <rotation>normal</rotation>
        <reflect-x>no</reflect-x>
        <reflect-y>no</reflect-y>
    </output>
    <output name="LVDS" screen="1">
        <vendor>X</vendor>
        <product>0x1234</product>
        <serial>0x12345678</serial>
        <width>1280</width>
        <height>800</height>
        <refresh-rate>60</refresh-rate>
        <rotation>normal</rotation>
        <reflect-x>no</reflect-x>
        <reflect-y>no</reflect-y>
    </output>
    <output name="VGA">
    </output>
</outputs>

Most of this should be rather self-explanatory (at least to me ;-) ). Just a few quick explanations:

• The naming is basically just supposed to reflect the naming in xrandr.
• The element <privacy>true</privacy> should inform the window-manager not to open any windows on that screen, as it would be a little annoying if any notification of new messages or similar would pop up on the projector during a presentation.
• The numbering of the screens is important for ie Plasma to load up the right Containment on the right screen.
• configurations marked as modifiable=false can not be altered, if the user wants to change the positions of the screens another (matching) configuration will be loaded instead

What this leaves open for now:

• What set of configurations will be shipped?
• Will it be possible create new configurations for the user?

Defaults

Defaults can be specified using wildcards:

<outputs configuration="extended-right">
    <output name="*" screen="0">
        <vendor>*</vendor>
        ...
    </output>
    <output name="*" screen="1">
        <vendor>*</vendor>
        ...
    </output>
</outputs>
<outputs configuration="extended-right">
    <output name="LVDS" screen="0">
        <vendor>*</vendor>
        ...
    </output>
    <output name="*" screen="1">
        <vendor>*</vendor>
        ...
    </output>
</outputs>

The best match would be used, so in this case, if LVDS exists, it will be screen 0 by default.

If it's possible to detect the type of plugged device, ie projector or plasma-tv ;-) the configuration could be extended to include defaults like this:

<outputs configuration="presentation">
    <output name="LVDS" screen="0">
        <type>*</type>
    </output>
    <output name="*" screen="1">
        <type>projector</type>
    </output>
</outputs>

Forcing outputs

For setups where the hardware/driver can not detect a device to be plugged in it might be necessary to force a certain output to be activated and to some mode. That could look like:

<configuration name="tv">
    <screen id="0">
    </screen>
    <screen id="1" force="composite">
        <privacy>true</privacy>
        <right-of>0</right-of>
    </screen>
</configuration>

<force name="composite" output="Composite">
    <width>768</width>
    <height>576</height>
    <rate>50</rate>
    <rotation>normal</rotation>
    <reflect_x>no</reflect_x>
    <reflect_y>no</reflect_y>
</force>

This could of course not be auto-selected, but would make it a lot easier for user to activate devices if needed.

This is not implemented yet.

Components / Naming

I decided to go with kephal as a name, so the components are named accordingly.

Base-Component / kephald

This should maybe changed to be a kded-module!

There should be a base-component, that will take care of the followings tasks:

• On startup check the connected devices and select a matching configuration.
• Publish informations such as primary or private
• Listen to XRandR events and switch configurations accordingly.
• Allow clients to select for what kind of events they want to be notified
  • Screen added/removed
  • Screen resized
  • Screen position changed
  • Rate changed
• Dispatch notifications to the clients
• Allow clients to change the selected configuration

This component should be able to fallback to XRandR 1.1/Xinerama if XRandR 1.2 is not available. Maybe it should even be able to fallback to using Qt's QDesktopWidget as its only source for information, that should allow this component to be used on all platforms. In that situation there would be no modifications possible, but it might still be interesting for applications that need to be informed of some screen-events.

Library / libkephal

This library wraps the d-bus api and provides an easy way to get the information needed.

Below is the current provided API:

Configurations

    /**
     * Configurations is the entrance-point to all
     * Configuration related operations.
     * Use: Configurations::instance() to obtain the currently
     * active instance.
     */
    class Configurations : public QObject {
        public:
            /**
             * Returns the currently active
             * instance.
             */
            static Configurations * instance();
            
            /**
             * Returns a list of all known Configurations.
             */
            virtual QMap<QString, Configuration *> configurations() = 0;
            
            /**
             * Returns the currently active Configuration.
             */
            virtual Configuration * activeConfiguration() = 0;
            
            /**
             * Returns a list of all alternate Configuratios
             * for the currently connected Outputs.
             */
            virtual QList<Configuration *> alternateConfigurations() = 0;
            
            /**
             * Returns the list of all positions in pixels
             * for the Output output.
             * These are the only positions that can be
             * passed to move().
             */
            virtual QList<QPoint> possiblePositions(Output * output) = 0;
            
            /**
             * Move Output output to position on the framebuffer.
             * This will relayout all Outputs.
             */
            virtual bool move(Output * output, const QPoint & position) = 0;
            
            /**
             * Resize Output output to size.
             * This will relayout all Outputs.
             */
            virtual bool resize(Output * output, const QSize & size) = 0;
            
            virtual bool rotate(Output * output, Rotation rotation) = 0;
            virtual bool reflectX(Output * output, bool reflect) = 0;
            virtual bool reflectY(Output * output, bool reflect) = 0;
            virtual bool changeRate(Output * output, float rate) = 0;
            
            /**
             * Find a Configuration by its name.
             * This returns 0 if the name is not known.
             */
            virtual Configuration * configuration(QString name);
            
            /**
             * Return the of the Screen this Output should
             * belong to.
             */
            virtual int screen(Output * output) = 0;
            
            virtual void setPolling(bool polling) = 0;
            virtual bool polling() = 0;
            
            virtual void confirm() = 0;
            virtual void revert() = 0;
            
        Q_SIGNALS:
            /**
             * This signal is emitted when the active
             * Configuration is changed.
             */
            void configurationActivated(kephal::Configuration * configuration);
            
            void pollingActivated();
            void pollingDeactivated();
            
            void confirmTimeout(int seconds);
            void confirmed();
            void reverted();
    };

Configuration

    /**
     * A Configuration allows to change settings that
     * control which Output belongs to which Screen or
     * is inactive.
     */
    class Configuration : public QObject {
        public:
            /**
             * The name of the Configuration.
             * This name uniquely identifies the
             * Configuration.
             */
            virtual QString name() = 0;
            
            /**
             * Returns whether this Configuration
             * can be modified.
             * This includes the layout and settings
             * like privacy-mode and primary Screen.
             */
            virtual bool isModifiable() = 0;
            
            /**
             * Returns whether this Configuration is
             * currently active.
             * Only 1 Configuration can be active at
             * any time.
             */
            virtual bool isActivated() = 0;
            
            /**
             * Returns the layout as Screens of size
             * 1x1.
             */
            virtual QMap<int, QPoint> layout() = 0;
            
            /**
             * Returns the id of the primary Screen for
             * this Configuration.
             */
            virtual int primaryScreen() = 0;
            
        public Q_SLOTS:
            /**
             * Activate this Configuration.
             */
            virtual void activate() = 0;
    };

Screens

    /**
     * Screens is the entrance-point for all Screen-
     * related operations.
     * Use: Screens::instance() for the currently
     * active instance.
     */
    class Screens : public QObject {
        public:
            /**
             * Returns the currently active instance.
             */
            static Screens * instance();
            
            /**
             * Returns the list of all current
             * Screens.
             * Every Screen has at least one Output
             * and a non-zero size.
             */
            virtual QList<Screen *> screens() = 0;
            
            /**
             * Find a Screen by its id.
             */
            virtual Screen * screen(int id);
            
            /**
             * Returns the current primary Screen.
             */
            Screen * primaryScreen();
            
        Q_SIGNALS:
            /**
             * This signal is emitted when a new
             * Screen appears, due to an Output
             * being activated or the Configuration
             * being changed.
             */
            void screenAdded(kephal::Screen * s);

            /**
             * This signal is emitted when a
             * Screen disappears, due to an Output
             * being deactivated or the
             * Configuration being changed.
             */
            void screenRemoved(int id);
            
            /**
             * This signal is emitted when the size
             * of the Screen changes.
             */
            void screenResized(kephal::Screen * s, QSize oldSize, QSize newSize);

            /**
             * This signal is emitted when the
             * position of the Screen changes.
             */
            void screenMoved(kephal::Screen * s, QPoint oldPosition, QPoint newPosition);
    };

Screen

    /**
     * A Screen is the area that is significant for
     * displaying windows.
     * Every activated Output belongs to exactly one
     * Screen and is completely contained within that
     * Screen. No 2 Screens overlap each other.
     */
    class Screen : public QObject {
        public:
            /**
             * Returns the id of this Screen. The id
             * is part of the Configuration and will
             * be the same whenever the same Outputs
             * are used with the same Configuration.
             */
            virtual int id() = 0;

            /**
             * The actual size of the screen in pixels.
             * This is the smallest area possible, so
             * that all Outputs are completely 
             * contained.
             */
            virtual QSize size() = 0;
            
            /**
             * The actual position on the framebuffer
             * in pixels.
             */
            virtual QPoint position() = 0;

            /**
             * Returns whether this Screen is to be
             * considered in privacy-mode.
             * In this mode no content should be
             * displayed on that screen unless the
             * user forces this.
             */
            virtual bool isPrivacyMode() = 0;
            
            /**
             * Sets the state of the privacy-mode.
             */
            virtual void setPrivacyMode(bool b) = 0;
            
            /**
             * Return a list of Outputs currently
             * being part of this Screen.
             */
            virtual QList<Output *> outputs() = 0;
            
            /**
             * Returns whether this screen is the
             * current primary screen.
             * This is just a convenience method,
             * since the real value is determined
             * the configuration used.
             */
            bool isPrimary();
            
            /**
             * Make this Screen the primary one.
             * This just calls the appropriate
             * method in the Configuration.
             */
            void setAsPrimary();
            
            /**
             * Returns the position and size of the
             * Screen as QRect.
             * This is just a convenience method.
             */
            QRect geom();
    };

Outputs

    /**
     * Outputs is the entrance-point to all Output
     * related operations.
     * Use: Outputs::instance() to obtain the currently
     * active instance.
     */
    class Outputs : public QObject {
        public:
            /**
             * Returns the currently active
             * instance.
             */
            static Outputs * instance();
            
            /**
             * Returns a list of all known Outputs,
             * even if they are inactive or
             * disconnected.
             */
            virtual QList<Output *> outputs() = 0;
            
            /**
             * Find an Output by its id.
             * Returns 0 if the id is not known.
             */
            virtual Output * output(const QString & id);
            
        Q_SIGNALS:
            /**
             * This signal is emitted when an Output
             * is connected.
             */
            void outputConnected(kephal::Output * o);

            /**
             * This signal is emitted when an Output
             * is disconnected.
             */
            void outputDisconnected(kephal::Output * o);

            /**
             * This signal is emitted when an Output
             * is activated.
             */
            void outputActivated(kephal::Output * o);

            /**
             * This signal is emitted when an Output
             * is deactivated.
             */
            void outputDeactivated(kephal::Output * o);

            /**
             * This signal is emitted when an Output
             * is resized from oldSize to newSize.
             */
            void outputResized(kephal::Output * o, QSize oldSize, QSize newSize);

            /**
             * This signal is emitted when an Output
             * is moved on the framebuffer from
             * oldPosition to newPosition.
             */
            void outputMoved(kephal::Output * o, QPoint oldPosition, QPoint newPosition);
            
            /**
             * This signal is emitted when the refresh-rate
             * of Output o changes.
             */
            void outputRateChanged(kephal::Output * o, float oldRate, float newRate);
            
            /**
             * This signal is emitted when the rotation of
             * Output o changes.
             */
            void outputRotated(kephal::Output * o, kephal::Rotation oldRotation, kephal::Rotation newRotation);
            
            /**
             * This signal is emitted when the reflection
             * state of Output o is changed.
             */
            void outputReflected(kephal::Output * o, bool oldX, bool oldY, bool newX, bool newY);
    };

Output

    /**
     * An Output is the actual connector and a
     * possibly connected monitor, such as
     * VGA, LVDS, TMDS-1.
     * This is most important for changing any
     * settings of the current setup.
     */
    class Output : public QObject {
        public:
            /**
             * Returns the id of the Output.
             * In case of XRandR 1.2 these will
             * be the names such as: VGA, LVDS,
             * TMDS-1.
             */
            virtual QString id() = 0;

            /**
             * Returns the actual size in pixels
             * if the Output is active.
             */
            virtual QSize size() = 0;

            /**
             * Returns the actual position in
             * pixels if the Output is active.
             */
            virtual QPoint position() = 0;
            
            /**
             * Returns whether this Output is
             * currently connected to a
             * monitor.
             */
            virtual bool isConnected() = 0;
            
            /**
             * Returns whether this Output is
             * currently activated and part
             * of a Screen.
             */
            virtual bool isActivated() = 0;
            
            /**
             * Returns a list of sizes, which
             * are supported by this Output.
             * This depends on the connected
             * monitor.
             */
            virtual QList<QSize> availableSizes() = 0;
            
            /**
             * Returns the vendor-code as
             * it is part of the EDID-block.
             */
            virtual QString vendor() = 0;
            
            /**
             * Returns the product-id as
             * it is part of the EDID-block.
             */
            virtual int productId() = 0;
            
            /**
             * Returns the serial-number as
             * it is part of the EDID-block.
             */
            virtual unsigned int serialNumber() = 0;
            
            /**
             * Returns the preffered size of
             * this Output. This depends on
             * the connected monitor.
             */
            virtual QSize preferredSize() = 0;
            
            /**
             * Returns the current rotation of
             * this Output.
             */
            virtual Rotation rotation() = 0;
            
            /**
             * Returns wether this Output is
             * currently reflected over the
             * x-axis.
             */
            virtual bool reflectX() = 0;
            
            /**
             * Returns wether this Output is
             * currently reflected over the
             * y-axis.
             */
            virtual bool reflectY() = 0;
            
            /**
             * Returns the current refresh-rate
             * of this Output.
             */
            virtual float rate() = 0;
            
            /**
             * Returns all possible refresh-rates
             * of this Output.
             * This depends on the connected
             * monitor.
             */
            virtual QList<float> availableRates() = 0;
            
            /**
             * This is just a convenience
             * method for looking up the 
             * Screen this Output belongs to.
             * Returns 0 if not active.
             */
            Screen * screen();
            
            /**
             * This convenience method
             * returns size and position of
             * this Output if active.
             */
            QRect geom();
            
            /**
             * Returns all available
             * positions for this Output.
             * This depends on the
             * Configuration used and
             * the positions of the other
             * active Outputs.
             */
            QList<QPoint> availablePositions();
            
        public Q_SLOTS:
            /**
             * This calls the appropriate
             * methods in the Configuration
             * to resize this Output.
             */
            bool resize(const QSize & size);

            /**
             * This calls the appropriate
             * methods in the Configuration
             * to move this Output.
             */
            bool move(const QPoint & position);
            
            /**
             * This will set this Ouputs rotation
             * to the given value.
             */
            bool rotate(Rotation rotation);
            
            /**
             * This will set this Output to be
             * reflected over the x-axis if reflect
             * is true.
             */
            bool reflectX(bool reflect);
            
            /**
             * This will set this Output to be
             * reflected over the y-axis if reflect
             * is true.
             */
            bool reflectY(bool reflect);
            
            /**
             * This will change this Outputs
             * refresh-rate to rate.
             */
            bool changeRate(double rate);
    };

Typical events

Plugging in a second unknown monitor

• Signal outputConnected is emitted
• If no configuration has been activated in response to that signal, a Configuration will be chosen based on the type (if available) of the connected monitor
• Defaults might be:
  • Regular monitor: Clone mode
  • Video-projector or TV: Presentation mode
  • Unknown: Inactive
• Signal outputMoved() is emitted for each output whose position changed
• Signal outputResized() is emitted for each output whose resolution changed
• Signal screenAdded() is emitted, if the result is a new screen
• Signal screenResized() is emitted for each screen that existed previously and whose resolution changed
• Signal screenMoved() is emitted for each screen that existed previously and whose position changed
• Signal activeConfigurationChanged() is emitted, if config actually changed

Plugging in a second known monitor

• Signal outputConnected is emitted
• If no configuration has been activated in response to that signal, the last used configuration with that monitor will be chosen
• Signal outputMoved() is emitted for each output whose position changed
• Signal outputResized() is emitted for each output whose resolution changed
• Signal screenAdded() is emitted, if the result is a new screen
• Signal screenResized() is emitted for each screen that existed previously and whose resolution changed
• Signal screenMoved() is emitted for each screen that existed previously and whose position changed
• Signal activeConfigurationChanged() is emitted, if config actually changed

Unplugging a monitor

• Signal outputDisconnected is emitted
• If no configuration has been activated in response to that signal, the last used configuration for the remaining monitor(s) will be chosen
• Signal screenRemoved() is emitted, if the new configuration has less screens
• Signal screenResized() is emitted for each screen whose resolution changed
• Signal screenMoved() is emitted for each screen whose position changed
• Signal activeConfigurationChanged() is emitted, if config actually changed

Handling of special setups

Overlapping outputs

If 2 (or more) outputs are overlapping, they will be treated as 1 big screen. This is just as broken behaviour as splitting them up into 2n-1 sub-screens or leaving parts "empty", but anything else would require major changes to other programs. So these kind of setups will just be considered broken and a proper positioning of outputs should be enforced.

Use-Cases

This will mostly be of use to users of laptops, but might also help others. Some possible users/cases are listed below.

Novice users

Users which do not know how to or dont want to waste time on setting up multi-head on their systems might find this to be easy enough to just give it a shot. For them everything needs to be as easy as possible and guided, thus not requiring to know "where to go". On plugging in a second screen they should be "asked" by their computer what to do, giving them advice, what each choice might be good for.

Users with a fixed number of different setups

This is what inspired me to work on this project. Some users will move between different setups with different requirements. This might be an external monitor at work, a different one at home and the video-projector in the meeting-room at work. For these users it is an absolute requirement that settings will be saved for each different setup, so that you get the same session when you get back to work as you left it the other day.

Users with lots of different setups

This might be people travelling and giving presentations a lot, but there are probably way more users, who keep using different monitors/projectors all the time. For those users it should be very quickly possible to change every aspect of the connected device, including resolution and refresh rate. This doesnt need to be a "1-click-action" though.

Special requirements for public screens

Some connected screens might be publicly visible, such as video-projectors for presentations. These should at no time contain any private data. This also includes things like e-mails and im's. In particular this means, that for newly connected screens clone-mode might not be the best idea, as this might expose private data until the mode is reconfigured.

Interaction with the user

In this section i want to explain some possible ways Plasma could react to such events and present them to the user.

The new device notifier

The new device notifier is contained in the panel for most users and deals with hotplugging of devices. So this should be the perfect place for an unobstrusive message about the connected monitor with some choices for easy configuration. An additional choice would be to open the advanced configuration tool.

On plugging in a monitor, it would be something like this:

• If the monitors are known, choose the appropriate configuration and do nothing else
• Otherwise, or if the notifier is clicked by the user, pop up the notifier, containing an entry for the monitor
• The following options should be available:
  • Clone mode
    • This would try to find the best matching resolution for both monitors and activate those
    • The user is then shown a passive popup with some (basic) information about the chosen resolutions and a link to change these
    • If he doesnt click it within x seconds the popup will fade out and the resolutions will stay as selected
  • Extended workspace, presentation mode and mediaplayer mode
    • In each case the user will be asked to click on the correct position in a little image within the popup
    • For extended workspace the monitor with the higher resolution will be chosen as the primary, for the other the previous monitor will stay the primary
  • Single-head mode
    • This simply turns off the previous monitor(s) and activates the newly plugged as only screen
  • Advanced configuration
    • An advanced configuration tool is shown

A dedicated screens plasmoid

This will visualize the connected monitors and the chosen configuration, allowing easy access to:

• Repositioning of monitors by drag and drop
• Changing configuration by overlayed icons
• Changing resolution by dragging a screen corner
• Changing refresh-rate by drop-down
• Selecting the primary monitor by clicking a little star

This plasmoid could then also be configured to not show anything unless multiple outputs are connected, thus not cluttering the interface unless it becomes useful.

Context menu

Users expect to be able to configure display settings from the context menu, so it should at least contain entries to launch the advanced configuration ui and to make the current screen the primary.

Desktop toolbox

Maybe the advanced configuration should even be accessible from the desktop toolbox.