Projects/KDE on Windows/Windows CE: Difference between revisions

From KDE TechBase
m (Console additional warning)
(rewritten memory section)
Line 22: Line 22:
===Memory limit===
===Memory limit===


using malloc I was not able to allocate more then 23MiB, although the limit should be at 32 MiB but with Virtual Alloc http://msdn.microsoft.com/en-us/library/aa908768.aspx it is possible to use every available bit of space on your device from one process using shared memory out of the process's slot.
The Memory limitations on Windows CE < 6 are a serious Problem. Additionally to the limitations that are normal for embedded development, you have to deal with limitations on the address room.  


''Before Windows Embedded CE 6.0 , if you call VirtualAlloc with dwSize >= 2 MB, flAllocationType set to MEM_RESERVE, and flProtect set to PAGE_NOACCESS, it automatically reserves memory at the shared memory region. This preserves per-process virtual memory.''
There is a very good Article about Windows CE < 6 Memory management and how to deal with the limitations at:
 
http://msdn.microsoft.com/en-us/library/ms836325.aspx
    void * ptr = VirtualAlloc(0,2*1024*1024,MEM_RESERVE,PAGE_NOACCESS);
    ptr = VirtualAlloc(ptr,2*1024*1024,MEM_COMMIT,PAGE_READWRITE);
 
did the trick for me.





Revision as of 10:48, 2 March 2010

Introduction

This page describes a work in progress, there are no KDE Applications running on Windows Mobile (that I know of). It is only intended to give a starting point to developers for cross compilation and Windows CE related issues. Since Qt is already ported for Windows CE it should be possible to get some KDE applications running on that Platform.

Windows CE / Windows Mobile

Although Windows CE (officially called Windows Embedded CE) is part of the Windows Embedded family there is a large difference between Windows Embedded CE and other Windows Embedded products. Most importantly Windows CE is not Windows. It has a different Kernel then any other Windows Operating System. And while other embedded products like Windows Embedded Standard or Windows Embedded Enterprise (which are sized down versions of Windows XP) will run Windows applications without problems, Windows CE applications require porting even from Windows.
Windows Mobile then is based on Windows CE (CE has no official meaning) and it's current Versions Windows Mobile 6.1 (and 6.5) are based on Windows CE 5.2 so we will be talking about developing for Windows CE 5.2 when we are talking about developing for Windows Mobile 6.1 or 6.5. It is confusing but Windows Mobile 6 is not based on Windows CE 6 so it still has the limitations of Windows CE 5. For example the process address space is still limited to 32 MiB and the maximum Number of processes can not exceed 32.


Process limit

Confirmed on HTC Touch Pro (Wimo6.5 CE 5.2.21854) running:

      for(i=0;i<32;i++){
         if(CreateProcess(L"test1.exe",(LPCWSTR)cptr,0,0,0,0,0,0,0,&procinf))
            fprintf(file,"Process Nr.%i ID: %u started\n",i,procinf.dwProcessId);
         else
            fprintf(file,"Creation failed after %i processes\n",i);
      }

I was able to create 16 processes at most after a fresh boot. (32 alltogether, including kernel thread)

Memory limit

The Memory limitations on Windows CE < 6 are a serious Problem. Additionally to the limitations that are normal for embedded development, you have to deal with limitations on the address room.

There is a very good Article about Windows CE < 6 Memory management and how to deal with the limitations at: http://msdn.microsoft.com/en-us/library/ms836325.aspx


Useful Windows Mobile Tools

Install the task manager before ssh. It is needed to kill the sshd process and to run ssh's adduser command.

Task Manager

http://www.dotfred.net/TaskMgr.htm is a very clean neat and good working task manager for Windows CE/Mobile it includes: Process manager ,CPU usage, Application Manager, Service Manager, Device Manager, Windows Manager, Notification Manager, IP Config utility, Ping utility, Net Stats utility, Registry Editor, Run program utility Really good to look behind the Windows Mobile curtain.

OpenSSH/Console

For OpenSSH we first need a Console on the device. There are two Console applications out there for Windows Mobile (that i know of). Both were originally developed for PocketPC 2003 OS both are dead Projects and both do not work very well on Windows Mobile 6.5. With the MS version i was only to input using t9 completion or copy and paste on my device, but it works with OpenSSH. PocketCMD worked "ok" on the device but went in an endless loop when using it from a ssh-connection.

Installing PPC Command Shell (from MS):

    Download and install (in Windows!) the Microsoft Windows Mobile Developer Power Toys
    Copy cmd.exe, console.dll, shell.exe in \windows on device
    Set HKEY_LOCAL_MACHINE/Drivers/Console/OutputTo to 0 

Warning: Setting Console OutputTo 0 will spawn Console Windows every time a program writes to stdout or stderr, using it with some Programs like HTC TouchFlow will also make your Device ridiculously slow.

There is a port for OpenSSH at http://www.codeplex.com/wikipage?projectname=CESSH The Project is dead though and since its use was intended to be an extension for MS Platform builder it is not documented for Windows Mobile. This recipe should get it working:

For OpenSSH:

    Download the Binaries
    Extract/Copy them onto the device
    Create a folder \Hard Disk on the device
    Generate the Host Keys
    Edit the sample sshd_config for your purposes
    Copy SocketToFile.dll, stftp.exe, scp.exe to the system base dir
    Taskmgr->Menu->Action->Run the adduser program and specify "username password" in the extra arguments input field
    Run sshd.exe

You can find the device IP address using the Task Manager.

You can then do things like

    ssh DEVICE /path/to/cmd.exe
    scp FILE DEVICE:/Speicherkarte

note that the first time you do these, you will need to confirm (on the device's screen) that running the unsigned programs (cmd, scp) from unknown publisher is allowed. So watch your device screen in case of problems!

Original Documentation can be found here: http://cessh.codeplex.com/wikipage?title=Compilation%20and%20use&referringTitle=Home

Notes: scp/sftp work great, ssh works enough to execute something on the device but not much more. Problem is with the Microsoft Pocket Console, i am looking for other Console implementations.

Symbolic tools PocketCMD/Console can be found here: http://www.walkingrandomly.com/?p=22

Registry Editor

There are a lot of registry editors for Windows CE.

http://ceregeditor.mdsoft.pl/ worked for me, although it is not Open Source it is small, clean, free of charge and does the job. It connects with the device and allows you to edit the Registry remotely.

A small registry editor is also included in the Task Manager.

Security Configuration Manager

Device Security Manager PowerToy for Windows Mobile 5.0

Usage is pretty self-explanatory just download it and executed on your host Windows machine with your mobile device connected to view and edit the security options. You can also use it to view the certificates installed on your device.


Building Qt for Windows CE with MSVC

Setting up Visual Studio

First thing you need is Visual Studio professional or standard edition (preferably with service pack 1). Be aware that the express editions are explicitly not supported for Windows CE development.
(You could use Microsoft Embedded VC which is cost-free, but depreciated and not supported by Qt)
Next thing you need is a Microsoft Windows Mobile Platform SDK corresponding to your target platform. Just search the Microsoft download pages or get the Windows Mobile 6 SDK from here (You should take a loot at the System Requirements)
If everything is set up correctly you should now be able to create a “Visual C++ smart device" project in Visual Studio.
Note on Version-Naming: Standard means without touchscreen, Professional means with Touchscreen and Classic means Windows Mobile without mobile phone capabilities

More about Visual Studio and Windows Mobile development: http://msdn.microsoft.com/en-us/library/dd721907.aspx

Building Qt

Download the Qt libraries for Windows CE from http://qt.nokia.com/downloads/win-ce-cpp and extract it, you may want to make sure that your path contains no blanks to avoid some problems.
Now you should add your <qt-directory>\bin to your Windows PATH environment variable (which can be done from “Control Panel → System → Advanced →Environment variables” )

Next get up your Visual Studio Command Prompt which should be accessible from the Windows Start Menu. Change the working directory to qt-directory and start the configuration. There should be a configuration.exe in your qt-directory with the usual options, (configure.exe -help for more info) for cross compiling you need to give at least the -platform parameter with your host system and -xplatform with your cross-compile target. Supported platforms are listed in the README file. e.g.:

configure.exe -platform win32-msvc2008 -xplatform wincewm60standard-msvc2008

Now that your configuration is set you can start the cross compilation. To switch from your “normal” build environment to the wince target Qt has a nice script included in the /bin directory (which should be in your PATH) just try calling setcepaths and provide your target configuration.
setcepaths wincewm60standard-msvc2008
With nmake (Visual Studio version of make) you can now build Qt for your target system.


Building Qt for Windows CE with CeGCC

Getting CeGCC

CeGCC (http://cegcc.sourceforge.net/) is a free cross-development environment for ARM-based systems running Windows CE. It is not officially supported by Qt and work with it is highly experimental, but given some work it could be possible to use it to build KDE-applications for Windows Mobile.

You should build cegcc from source, see: http://cegcc.sourceforge.net/docs/build-toolchain.html the current release 0.59 has some major bugs which were fixed since.


Building Qt

Maurice Kalinowski at Nokia did some work on this and provided a working Qt-Version with make specs for mingw32ce that make it possible to build most of the Qt parts. The original mailing-list thread discussing this can be found here: http://lists.trolltech.com/pipermail/qt-wince-interest/2009-November/000764.html

Link to the gitorious repository: http://gitorious.org/+qt-developers/qt/ce-gcc

After downloading you should take a look at the cegcc_configure script, maybe change the installation prefix there and then execute it. If your configuration fails make sure that your environment is not yet set for cross-compiling.

When configuration is complete you can build the host-tools, just go into your checkout and execute:
{

       QTCEDIR=$(pwd)
       folders="bootstrap moc uic rcc"
       for f in $folders; do
           cd $QTCEDIR/src/tools/$f && $QTCEDIR/bin/qmake -spec $QTCEDIR/mkspecs/linux-g++ && make && cd $QTCEDIR || return
       done

}

If the tools were built you now need to set up your environment for cross-compiling.

        export MINGWPATH=/opt/mingw32ce
        export INCLUDE=$MINGWPATH/arm-mingw32ce/include:$MINGWPATH/include:$INCLUDE
        export LIB=$MINGWPATH/arm-mingw32ce/lib:$MINGWPATH/lib:$LIB
        export PATH=$MINGWPATH/bin:$MINGWPATH/arm-mingw32ce/bin:$MINGWPATH/libexec/gcc/arm-mingw32ce/4.4.0:$PATH

Of course you have to change MINGWPATH according to your installation.

Now all that is left to do is to build Qt for the target system, to do that (again from your checkout directory) execute:
{

       QTCEDIR=$(pwd)
       folders="winmain corelib gui"
       for f in $folders; do
           cd $QTCEDIR/src/$f && $QTCEDIR/bin/qmake && make && make install && cd $QTCEDIR || return
       done

} This will build the qtcore and qtgui, you can try building more parts, of course.


Notes:

- in case of pch errors delete .pch/release-static-emb-windowsce/qt_gui_pch.h.gch directory

- when you are compiling your application outside of the Qt tree, you might get into trouble related to mkspecs and include dirs.

- for deployment you need to copy these files also to the same directory on the device:

   libgcc_s_sjlj-1.dll
   libstdc++-6.dll
 which are located inside the arm-mingw32ce directory


Packaging:

Deploying Qt Windows Mobile Applications using LCAB

LCAB is a free software utility, that has the ability to create Microsoft CAB archives. You can get it on ( http://packages.debian.org/lenny/lcab )or as a Debian package.

With the creation of cabinet archives as a prerequisite, the question is how to get Windows Mobile to recognize them as install packages. Its basically a way in which the files have to be ordered and named inside the archive combined with a setup.xml file with installation instructions. There is a Perl script to add the needed manifest and setup configuration file. You can find it under ( http://files.intevation.de/incoming/aheinecke/cab_files/pocketpc-cab.pl )

This script needs a configuration file (two if you want to install registry keys, also), but it is much more simple and straightforward then the Cabwiz variant. Simply write down for every file you want to package a line a file:

   Filename Target-Directory [Shortcut Name] [Shortcut Folder] 

for example:

   collidingmice.exe %CE1%/Colliding_Mice "Colliding Mice" %CE11% 
   msvcr90.dll %CE2% 
   QtCore4.dll %CE2% 
   QtGui4.dll %CE2% 

This installs collidingmice.exe to Program Files/Colliding_Mice , places a shortcut in the Start Menu folder and copies the libraries into the Windows directory.

CE Variables

Here are some predefined Windows CE variables which can be used to identify a path on a Windows CE device.

      %CE1%	\Program Files
      %CE2%	\Windows
      %CE4%	\Windows\StartUp
      %CE5%	\My Documents
      %CE6%	\Program Files\Accessories
      %CE7%	\Program Files\Communication
      %CE8%	\Program Files\Games
      %CE11%	\Windows\Start Menu\Programs
      %CE12%	\Windows\Start Menu\Programs\Accessories
      %CE13%	\Windows\Start Menu\Programs\Communications
      %CE14%	\Windows\Start Menu\Programs\Games
      %CE15%	\Windows\Fonts
      %CE17%	\Windows\Start Menu

Deploying Windows Mobile Applications using Cabwiz

Maybe in some situations Cabwiz has its relevance, (e.g. you only want to build from windows) but even then i would suggest to directly edit the setup.xml that is included in the Cabinet file to suit your needs. Before i found lcab i wrote a guide on how to Package a simple Qt-Application with cabwiz

Building KDE-Apps for Windows CE using MSVC

Note: It has not yet been done, these are just some ideas.

Entry points for CE CRT:

     Your entrypoint      CRT entrypoint
        main                  mainACRTStartup
        wmain                 mainWCRTStartup
        WinMain               WinMainCRTStartup
        wWinMain              wWinMainCRTStartup
        DllMain               _DllMainCRTStartup

Problems with MSVC

If you try to port native code from Linux to Windows CE or even from Windows to Windows CE you will soon run into walls because Microsoft does not provide you with a C-Library that is in following any standards. This is ugly when some parts are missing like errno, fcntl, types, stat, ... but gets even more ugly when there are the headers, but not the library functions.

Compatibility Libraries

The idea is not to port the software to Windows CE/Mobile, but to instead to expand the Windows CE SDK in a way that it supports most of the functions Windows does. Although Windows Mobile SDK's still are a subset af the Windows libraries they came a long way in the last years, and most of the functions that are missing seem to be on a basic level. Maybe it is more of an annoyance then an actual problem.

CELIB

With all the Problems there are with Windows CE C-Libraries it was actually much worse in older Windows CE versions. Rainer Keuchel did a lot of porting back in 2003 ( http://www.rainer-keuchel.de/software.html ) and wrote celib which provides many of the functions Windows CE is lacking. Sadly, this project died and has not been maintained to fit current Windows Versions. But maybe it can help with some ideas for workarounds or to cannibalize some parts.

wcelibcex

Wcelibcex supplements the Windows CE C-library with functions and workarounds for porting applications from Unix and Windows to Windows CE. Originally, the WCELIBCEX development was founded by Taxus SI Ltd. and in 2005 released as an Open Source Software project. The WCELIBCEX project is currently maintained by Mateusz Łoskot. ( http://wcelibcex.sourceforge.net/ )

At a first glance this project is still active, at least the SVN Version is updated now and then. I've had some success using it but it also is not as complete as one would hope. There are some interesting ideas implemented, like managing the environment through the Windows registry. There would is still a lot to include if we wanted to build without modifying the public libraries but it would give a good starting point.

Installing wcelibcex

As mentioned the release Version is outdated, so get you should get wcelibcex from svn

       c:\>svn co https:/ / wcelibcex.svn.sourceforge.net/svnroot/wcelibcex/trunk wcelibcex

Open it in Visual Studio, select your target configuration and build. Make sure to include wcelibcex in your include and lib paths when you want to use it.



Dependencies

Following the Getting Started/Build/KDE4/Windows/Building DBus guide we need at least the following libraries.

   * aspell  
   * boost-headers
   * bzip2
   * cyrus-sasl
   * expat
   * giflib
   * gpgme
   * iconv
   * jasper
   * jpeg
   * kdewin32
   * libintl
   * libpng ( ported V 1.2.3 )
   * libxml2
   * libxslt
   * openldap
   * openslp
   * openssl ( ported 0.9.8a )
   * pcre
   * qca2
   * qimageblitz 
   * redland
   * strigi
   * soprano
   * tiff
   * update-mime-database
   * windbus
   * zlib ( ported V 1.1.4 )


Building Library Dependencies

This part now really becomes 'read at own risk' at this point I have no idea if this is the best way to build or the easiest way to build or even a slightly sane one. My goal was to come up with a way to compile free software that has been ported for Windows to Windows CE without changing the code but rather by finding workarounds for all the portability issues. I used the before-mentioned wcelibcex to try to get some libraries to run on my Windows Mobile device. For building i used nmake makefiles and no Visual Studio Project files to have more control over the build.


Setting up the Environment

To Cross-Compile with the MSVC compiler you should write yourself a script to set up your environment, if you don't want to set your system wide variables you'd have to enter really long commands every time you open a new console by hand. It could be looking like this (in a standard installation):

     @echo off
     call setcepaths wincewm60standard-msvc2008
     set INCLUDE=%INCLUDE%;C:\wcelibcex\src
     set LIB=%LIB%;C:\wcelibcex\msvc90\Windows Mobile 6 Standard SDK (ARMV4I)\Release
     set _CL_=/DWIN32 /D_WIN32 /D_WIN32_ /D_WINDOWS /D_WIN32_WCE#0x502 /DUNDER_CE#0x502 /DWINCE 
/DARMV4I /D_ARMV4I_ /D_ARM_ /D_M_ARM /DARM /D__arm__ /D_UNICODE /DUNICODE set _LINK_=corelibc.lib coredll.lib wcelibcex.lib set LINK=/entry:"_DllMainCRTStartup" /nodefaultlib:oldnames.lib subsystem:WINDOWSCE

Qt offers a script for most of the INCLUDE, LIB, and PATH variables, called setcepaths, in this example i used it to get the cross-compile paths initialized. Please note the 0x502 Version Numbers are intentional when developing for Windows Mobile 6 we are compiling for Windwos CE 5.2 See: Windows CE/Windows Mobile
The # character replaces = for the Windows Environment variables. As you can see I also included wcelibcex since I want to demonstrate how to use the wceex-functions and need it for errno.h.

CL,_CL_,LINK,_LINK_ are MSVC variables to control cl.exe and link.exe CL is prepend to all options when cl.exe is called,_CL_ is appended, same goes for link.exe and its variables. Note: When using defines cl.exe always uses the last definition it got. So with _CL_ you could override defines from any cl call (e.g. a makefile).


Building libiconv

The last version I have found with msvc Makefiles for iconv was 1.11.1 for a first try to get it working I used that version to have it easier.
get it at: http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.11.1.tar.gz extract and then run:

           set CL=/Dabort#wceex_abort /Dperror(...)#printf(...) /Dstrdup#_strdup /Dstrerror#wceex_strerror

edit the config.h.msvc (or just put #undefs at the end of the file):

            #define HAVE_ENVIRON_DECL 1 to #undef HAVE_ENVIRON_DECL
            #define HAVE_SETLOCALE 1 to #undef HAVE_SETLOCALE

now run

            nmake -f Makefile.msvc NO_NLS=1 DLL=1 MFLAGS=-MD
            nmake -f Makefile.msvc NO_NLS=1 DLL=1 MFLAGS=-MD install

Explanation: The CL defines reroute some function calls. We don't have decent error handling on Windows CE so in my opinion we could just print out error messages or numbers to a console we also don't have (but maybe later find a way to reroute to a file or so) anyways. wceex_abort just terminates the process _strdup is the WinCE equivalent of strdup, MS already provided it. I did not use more configuration Options in config.h because in most cases the workarounds they provide don't help either for Windows CE. NO_NLS=1 stands for building without libintl, since libintl needs iconv to build.


Building gettext-runtime

With gettext I again used a release with msvc makefiles, version 0.14.5 http://ftp.gnu.org/pub/gnu/gettext/gettext-0.14.5.tar.gz

If you have installed libiconv in the default location you can also use defaults with gettext, otherwise make sure to provide the same prefix you used with libiconv. To avoid confusing errors, you may want to edit the makefile to only build the gettext-runtime. Now in the gettext-runtime directory append to the config.h.msvc again:

     #undef HAVE_ENVIRON_DECL
     #undef HAVE_SETLOCALE

and change CL to:

     CL=/Dabort#wceex_abort /Dperror(...)#printf(...) /Dstrdup#_strdup /Dstrerror#wceex_strerror 
/Dgetenv#wceex_getenv /Dclose#wceex_close /Dread#wceex_read
/Dfstat#wceex_fstat /Dopen#wceex_open /Dbsearch#wceex_bsearch /Draise#wceex_raise /Dgetcwd#wceex_getcwd
/DGetThreadLocale#GetUserDefaultLCID

loadmsgcat needs close read write fstat open bsearch and raise. Since there were no implementations or workarounds for close read fstat and open in wcelibcex I wrote them myself using fopen fclose. For wceex_raise I took parts from Rainer Keuchel's celib. I'll try to do some decent testing and getting them upstream. But if you have an idea how not to use them please make an edit.

Now you should be able to build:

      nmake /f Makefile.msvc DLL=1 MFLAGS=-MD
      nmake /f Makefile.msvc DLL=1 MFLAGS=-MD install

If you get errors make sure that you do not try to build the tools, which are another problem.

Building libiconv with libintl

Go back to your libiconv directory

      nmake /f Makefile.msvc DLL=1 MFLAGS=-MD clean
      nmake /f Makefile.msvc DLL=1 MFLAGS=-MD
      nmake /f Makefile.msvc DLL=1 MFLAGS=-MD install

should run smoothly and link against libintl. If not, make sure that you still have the preprocessor macros defined in CL.

libxml2 with iconv

Building libxml2 for WinCE is fairly easy for us, because the official release already was ported to Windows CE, my problem was that I could not convert the EVC project files to my VS installation so I used the Win32 makefile. Checkout libxml2

     git clone git://git.gnome.org/libxml2

Although a port exists it was apparently done some time ago so you have to modify some things, the file include\wsockcompat.h includes winsock.h instead of winsock2.h for _WIN32_WCE. That is no longer needed we have winsock2 on Windows Mobile 6, so change that to winsock2.h
Copy the two files wincecompat.h and wincecompat.c from win32\wince into your main src directory in those all the functions needed for compatibility are declared. To avoid conflicts make sure your CL variable is not still set with all the function replacements. Since the original port used EVC-Project files we have to edit the win32\Makefile.msvc to avoid linker errors.

     In line 69 replace wsock32.lib ws2_32.lib with ws2.lib
     In line 81 replace kernel32.lib with coredll.lib 
     Add $(XML_INTDIR)\wincecompat.obj to the XML_OBJ list. 

Now from the win32 subdirectory you should run:

     cscript configure.js help

and configure according to your setup. Make sure to set the inc/lib vars pointing to your iconv installation.

     nmake /f win32\Makefile libxml libxmla libxmladll

should build. Only some utils are still going to make problems.


Miscellaneous

Accessing Pocket Outlook

Similar to the Outlook Object Model for the Desktop Outlook, Microsoft offers the Pocket Outlook Object Model (POOM) The related library is called pimstore and part of the Windows Mobile SDK. You can use it to access the pim.vol file which is located in the Windows Mobile root directory. It contains Contacts, Tasks, Calendar entries and the infrared folder. To access messages you have to use the Messaging API.

Establishing a connection is straightforward COM, you have to initialize the COM library find out the COM identifier of PocketOutlook, then create an application class COM server object and logon to the COM server object using the IPOutlookApp COM interface

For example:

   CLSID clsid;  // The COM Identifier 
   CComQIPtr<IPOutlookApp,&__uuidof(IPOutlookApp)>                            ipOutlookAppPtr; // managed COM interface pointer
   
   CoInitializeEx(0,0); // Initialize the COM library
   CLSIDFromProgID(L"PocketOutlook.Application", &clsid); // Lookup the ID in the registry
   ipOutlookAppPtr.CoCreateInstance(clsid, NULL, CLSTX_INPROC_SERVER); // Create the server object
   ipOutlookAppPtr->Logon(NULL); // Logon

After you have established the connection you can use the IPOutlook interface methods to retrieve or manipulate the data. I have written a small program that exports all the Contact data on a device into a CSV file. POutlookExport.zip

Further Reading: MSDN Messaging Reference MSDN POOM Reference