Integrating Sublime Text with GPII

Author: Christophe Strobbe.

Licence: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0).

Warning: this tutorial was written in 2016 and is now outdated.

Introduction

This document describes how you can integrate a dekstop application with GPII's real-time framework. The programming editor Sublime Text (version 3 for Windows) is used as an example. For a general introduction to GPII and the integration process, see GPII/Cloud4all for Developers and the GPII wiki pages referenced in that document. For an similar but older document, see Building A Simple Solution on the GPII wiki.

Installing GPII on Windows

The process described below is based on a 64-bit version of Microsoft Windows 7 but should also work on a 32-bit version of Windows 7 and on more recent versions of Windows.

Installing Dependencies for Windows

You need the following dependencies for installing GPII on Windows:

• Git: install both GitHub Desktop for Windows and Git for Windows (sometimes referred to as “msysgit”).
• Node.js: install the 32-bit version of Node.js 4.x, even if you are using a 64-bit version of Windows. (After installing, you can check Node.js by typing node --version on the command line.)
• Visual Studio 2013: install Visual Studio Express 2013 for desktop or Visual Studio Community 2013. (Visual Studio 2015 Express might also work, but we have not been lucky with it.)
• Visual C++ Redistributable Packages for Visual Studio 2013.
• Python 2.7.x: Python 2.7.11. (Python is only needed to install and build GPII, not to run it.) After installing Python, go to the operating system's environment variables and add the Python directory to the PATH variable. (You can check your Python installation by typing python --version on the command line. We have not checked whether Python 3.x also works. However, node-gyp does not work with Python 3.x.)

If you run into problems installing any of these dependencies, check the Windows installation instructions on the GPIi wiki.

Installing the GPII Real-Time Framework

Open a command line window, go to the directory where you want to intall GPII and type the following commands:

• mkdir gpii
• cd gpii
• git clone git://github.com/GPII/windows.git
• cd windows
• npm install
• npm install grunt
  Note: this step may fail; it is currently not clear wether it is necessary.
• npm install -g grunt-cli
• grunt build
• You can now check whether GPII works. Start GPII: node gpii.js
  Now test whether the personalisation system works by entering the following URL into a browser:
  http://localhost:8081/user/sammy/logonChange.
  This “logs in” the user “sammy”, i.e. applies Sammy's needs and preferences to Windows. As a result, the Windows magnifier should launch and the user interface colours should be inverted. Use the same URL again to undo the adaptations:
  http://localhost:8081/user/sammy/logonChange.
• As an additional check, you can also run the acceptance tests: go to the directory gpii/node_modules/universal/tests/ and enter the command node IntegrationTests.js. When the integration tests have finished, you should see a message like the following: jq: All tests concluded: 190/190 total passed in 50981 ms - PASS.

Integrating Sublime Text 3

Identifying Sublime Text 3's Executable and Settings File

First, download and install Sublime Text (version 3). Then, note the two following things:

1. Where the user settings are stored. GPII needs to know this location, so it can adapt the settings.
   The path to Sublime Text's settings file typically has the following pattern: C:\Users\<user>\AppData\Roaming\Sublime Text 3\Packages\User\Preferences.sublime-settings (where <user> represents the current user account).
   It is possible that this file does not exist immediately after installation. In order to create the file, you can start Sublime Text and change one of its settings, such as the colour scheme or the font size (in the Preferences menu), or the settings related to spelling in the View menu. Alternatively, you can go to Preferences > Settings – User and edit the settings file directly. After this, you should find a settings file at the location mentioned above.
2. How the location of the Sublime Text executable is represented in the Windows registry. GPII needs to know this location, so it can start and stop the application.
   You can find this path by searching for the name of the executable, sublime_text.exe, in the Windows registry (using regedit). The Windows registry entry for Sublime Text can be found at HKEY_CLASSES_ROOT\Applications\sublime_text.exe.

Integration Steps for Sublime Text 3

Identifying Sublime Text's Settings

The next step is the identification and description of Sublime Text's settings. Sublime Text has a hundred different settings; they can be found in the JSON-like file at C:\Users\<user>\AppData\Roaming\Sublime Text 3\Packages\Default\\Preferences.sublime-settings. (The file is not pure JSON because JSON syntax does not allow JavaScript comments.) Since GPII/Cloud4all currently does not have a database for registering applications with all their settings, we will focus on those Sublime Text settings that are relevant to accessibility or that we want to show in demonstrations (or both). For each of these settings, we provide the following types of information: its internal name, its data type, its value range, its default value (if applicable), and any dependencies on other settings (for example, the dictionary setting is only relevant when the spell_check setting has the value true).

In addition, we also create a solution ID for Sublime Text, i.e. a string that will uniquely identify the application in GPII. Based on the URL of the website where Sublime Text can be downloaded, we give the application the following ID: com.sublimetext. The following table summarizes the data involved in this step. (Setting are listed in alphabetical order. The column “common term” will be explained below.)

Each of the application's settings can be uniquely identified using the following pattern: http://registry.gpii.net/applications/ + solution ID + setting name. For example, the Sublime Text setting caret_extra_bottom is uniquely identified in GPII by the following URI: http://registry.gpii.net/applications/com.sublimetext/caret_extra_bottom. The category of settings represented by means of this type of URI is known as application-specific terms.

The last column in the settings table represents the common terms. The Cloud4all project created a set of common terms that represent an application-neutral way of representing settings. Common terms serve as a lingua franca for representing settings without relying on application-specific terms. The have a data type and value space that can be easily translated to application-specific terms. For example, the common term speechRate (data type: integer) represents the speed of a text-to-speech engine's output in words per minute. This type of value can be translated or “transformed” (see below) to the application-specific setting for speech rate in various screen readers.

The Cloud4all project created a small set of 50 common terms, and most of Sublime Text's settings have no corresponding common term. Sublime Text's font_size setting (an integer representing size in pixels) can easily be represented by the common term fontSize, which represents a size in points. However, the common terms highContrastEnabled (Boolean) and highContrastTheme (with the possible values black-white, white-black, yellow-black, black-yellow, lime-black) were conceived for users who need high-contrast settings, while Sublime Text's colour schemes are much more complex and were not conceived to represent high-contrast settings. In fact, many of Sublime Text's colour schemes are low-contrast schemes.
In summary, Sublime Text's settings, with the exception of font_size, can currently only be represented using application-specific terms.

Creating a Solutions Registry Entry and Related Files

The integration of a solution requires modifications to a few existing files and the creation of few new files:

• win32.json: this is the Windows solutions registry. For each solution that runs on Windows, the registry describes where the solution's settings can be found, how the solution can be launched and stopped, how common terms (if any) can be “transformed” to the solution's application-specific terms, and a few other things.
• installedSolutions.json: this is the list of integrated solutions that the GPII's device reporter can report as available. (This is currently a list that needs to be edited manually; later, the list of solutions available on a system should be reported automatically by a component known as the Available Solutions Scanner.)
• A description of the solution (in Markdown format).
• A number of preference sets that can be used to demonstrate the integration of the solutions, to be stored in the directory gpii/node_modules/universal/testData/preferences/.
• Several files for running acceptance tests or integration tests (discussed below). Without such tests, the pull request for the solution integration will not be merged into the GPII code repository. The tests are run as part of GPII's continuous integration system.

Note: Before modifying or creating these files, you should first create a new branch in your fork of GPII's “universal” repository. This branch will contain all the changes and additions that are required to integrate the application with GPII, including the integration tests, which are explained below. Ideally, the name of the branch matches the name of the JIRA ticket (in GPII's issue tracking system) for the solution integration.
For Sublime Text, the ticket for the Windows integration is GPII-1796, so the corresponding branch is also called GPII-1796.

The entry for Sublime Text in the Windows solutions registry win32.json looks as follows:

"com.sublimetext": {
    "name": "Sublime Text",
    "contexts": {
        "OS": [
            {
                "id": "win32",
                "version": ">=5.0"
            }
        ]
    },
    "settingsHandlers": {
        "configuration": {
            "type": "gpii.settingsHandlers.JSONSettingsHandler",
            "options": {
                "filename": "${{environment}.APPDATA}\\Sublime Text 3\\Packages\\User\\Preferences.sublime-settings"
            },
            "capabilities": [
                "applications.com\\.sublimetext\\.id",
                "caret_extra_bottom",
                "caret_extra_top",
                "caret_extra_width",
                "caret_style",
                "color_scheme",
                "dictionary",
                "fade_fold_buttons",
                "font_face",
                "font_size",
                "gutter",
                "highlight_line",
                "highlight_modified_tabs",
                "line_numbers",
                "margin",
                "show_encoding",
                "show_line_endings",
                "spell_check",
                "translate_tabs_to_spaces"
            ],
            "capabilitiesTransformations": {
                "font_size": {
                    "transform": {
                        "type": "fluid.transforms.round",
                        "input": {
                            "transform": {
                                "type": "fluid.transforms.linearScale",
                                "inputPath": "http://registry\\.gpii\\.net/common/fontSize",
                                "factor": 1.33
                            }
                        }
                    }
                }
            }
        }
    },
    "configure": [
        "settings.configuration"
    ],
    "restore": [
        "settings.configuration"
    ],
    "start": [
        {
            "type": "gpii.launch.exec",
            "command": "\"${{registry}.HKEY_CLASSES_ROOT\\Applications\\sublime_text.exe\\}\""
        }
    ],
    "stop": [
    ],
    "isInstalled": [
        {
            "type": "gpii.deviceReporter.alwaysInstalled"
        }
    ]
}

The solution entry starts with Sublime Text's solution ID (see above) and contains the following sections:

• name: the solution's human-readable name.
• contexts: the platform and versions on which the solution runs. win32 obviously denotes Microsoft Windows. >=5.0 means “version 5 or higher”. This refers to the internal version numbering, where version 5.1 refers to Windows XP, version 6.0 refers to Windows Vista, version 6.1 refers to Windows 7, etc. (based on Wikipedia's list of Windows versions).
• settingsHandlers contains several important parts:
  • The settingsHandler block configuration describes two types of data. type identifies the type of settings handler that is needed to read and write the solution's settings. Since Sublime Text uses JSON to store its settings, we use the GPII's built-in code for dealing with this format. This settings handler is identified as gpii.settingsHandlers.JSONSettingsHandler. (See the wiki page Settings Handler for more background information.)
    filename describes the path to the settings file. In this path, ${{environment}.APPDATA} is used to refer to the APPDATA directory associated with the account of the user who is logged in to Windows whilst GPII is running. (${{environment}.APPDATA} resolves to the path that is returned when you enter echo %APPDATA% on the command line.)
  • capabilities: consists of a different version of the solution's identifier followed by a list of application-specific terms that GPII should recognise. This is potentially the entire list of application settings.
  • capabilitiesTransformations is a section with transformations that converts settings expressed as common terms (if any are present in the user's preference set) into application-specific settings. The length of this section depends on the number of common terms that can be mapped to the solution's application-specific terms. In the case of Sublime Text, there is only one common term that can be mapped to any of the solution's application-specific terms, i.e. fontSize.
    The transformations are expressed in a declarative format. For more details, see the wiki page Architecture - Available transformation functions.
• configure and restore have to do with the mechanisms that GPII uses to write to the solution's persistence system for settings (JSON, XML, the Windows registry, etc.) and how the original settings should be restored once the “GPII user” logs out from GPII on the device (PC, smartphone, …). The values in these blocks are either references to existing “settingsHandlers blocks” or a customised lifecycle block. For Sublime Text, we use the value settings.configuration. Note that the last part of this value, configuration, matches the name “configuration” we gave to the settingsHandler block (above).
• start and stop refers to how the solution should be launched (if it is not already running) and stopped (if it was not running before the “GPII user” logged in).
• isInstalled contains mechanisms to detect whether the solution is installed.

Next, you need to add an entry for the solution to the file testData/deviceReporter/installedSolutions.json. For Sublime Text, the following addition was made:

{
  "id": "com.sublimetext"
}

You should also create a few needs & preferences sets that you can use to demonstrate the adaptations that GPII can make to the solution's settings. These needs and preferences sets (short: NP sets) are also in JSON. NP sets can use common terms, application-specific terms or a combination of these. For example, the NP set sublime_carla uses the common term fontSize and several application-specific terms for Sublime Text:

{
    "flat": {
        "contexts": {
            "gpii-default": {
                "name": "Default preferences",
                "preferences": {
                    "http://registry.gpii.net/common/fontSize": 24,
                    "http://registry.gpii.net/applications/com.sublimetext": {
                      "caret_extra_bottom": 1,
                      "caret_extra_top": 1,
                      "caret_extra_width": 4,
                      "color_scheme": "Packages/Color Scheme - Default/Blackboard.tmTheme",
                      "dictionary": "Packages/Language - English/en_GB.dic",
                      "highlight_line": true,
                      "highlight_modified_tabs": true,
                      "spell_check": true
                    }
                }
            }
        }
    }
}

A solution's NP sets should always be accompanied by a Markdown file that identifies the solution they were created for and that explains the intended effect of the NP sets. The file names for both the NP sets and the Markdown file should have a prefix that is unique to the solution, e.g. sublime_ for Sublime Text. (This is not required by the system; it is only meant to make it easier to figure out which NP sets belong to which solution.) These files are all stored in the directory universal/testData/preferences/.

Finally, you should also create a Markdown file that describes the solution, the settings that have been integrated, the NP sets used for the acceptance tests (see below) and any other information that the GPII developer team would need to know when reviewing your contribution.

At this point, you should already be able to test whether the integration works. Open a command line, navigate to the windows directory in your GPII installation and start GPII by typing node gpii.js. Open Sublime Text. Then enter a URL like the following into a browser: http://localhost:8081/user/sublime_carla/logonChange. (sublime_carla is an NP set that we created for this kind of test.) Sublime Text should now adapt to the settings defined in the NP set sublime_carla. (If you opened the settings file Preferences.sublime-settings before this test, you will be able to see the resulting settings there.) Then activate the same link again to “log out” sublime_carla; GPII should now restore the original settings.

Writing Integration Tests

The instruction in this section are based on the wiki page Writing Acceptance Tests, more specifically the section Writing Acceptance Tests for Applications Using the Local Flow Manager.

The integration tests are part of GPII's automatic acceptance testing framework. The acceptance tests for a specific solution test the entire real-time framework end to end. Simply put, the tests start up the GPII server, log in a user, and checks that the system is modified according to that user's needs & preferences set. It then logs that user out again, restores the system to its original settings and checks that the system has been restored to its original state. This type of tests also includes testing of the transformations (see above).

First, you need to create a few needs & preferences sets (NP sets) that you want to test your application with. These should be stored in the directory universal/testData/preferences/acceptanceTests/. You can reuse the NP sets that you have already created for demonstrating the solution.

You should also create a device reporter file, which is to be stored in the directory universal/testData/deviceReporter/acceptanceTests/. The device reporter file name should be descriptive of its content, e.g. sublimetext.json, and its content looks like the following example:

[
  {
    "id": "com.sublimetext"
  }
]

The next step is to create the configuration file that the testing framework should use when running the integration tests for the application. The purpose of this file is to point the testing framework to the device reporter file created in the previous step.

Since our example is the Windows version of Sublime text, the configuration file should be stored in the directory universal/tests/platform/windows/configs/. The file name uses the pattern gpii.tests.acceptance.windows.<application>.config.json, where <application> stands for the application being integrated. For Sublime Text, the config file should be called gpii.tests.acceptance.windows.sublimetext.config.json. Its content looks as follows:

{
    "type": "gpii.tests.acceptance.windows.sublimetext.config",
    "options": {
        "distributeOptions": {
            "acceptance.installedSolutionsPath": {
                "record": "%universal/testData/deviceReporter/acceptanceTests/sublimetext.json",
                "target": "{that deviceReporter installedSolutionsDataSource}.options.path",
                "priority": "after:development.installedSolutionsPath"
            }
        }
    },       
    "mergeConfigs": "%universal/tests/configs/gpii.tests.acceptance.localInstall.config.json"
}

The important part of this file is the line starting with record, which points to the device reporter file. In addition, value of type in the second line reflects the configuration file name without its file name extension.

The final step is to create the file with the actual acceptance test(s). Acceptance tests for Windows solutions are stored in the directory universal/tests/platform/windows/. Each of the transformations in the solution's registry entry should have at least one unit test; it is good practice to cover some edge cases for the common terms that can be mapped to your solution's application-specific terms. The acceptance tests for Sublime Text on Windows are shown below by way of example:

"use strict";
var fluid = require("universal"),
    gpii = fluid.registerNamespace("gpii");

gpii.loadTestingSupport();

fluid.registerNamespace("gpii.tests.windows");

gpii.tests.windows.sublimetext = [
    {
        name: "Acceptance test for small font size in Sublime Text",
        userToken: "sublime_font8",
        settingsHandlers: {
            "gpii.settingsHandlers.JSONSettingsHandler": {
                "data": [
                    {
                        "settings": {
                            "font_size": 11
                        },
                        "options": {
                            "filename": "${{environment}.APPDATA}\\Sublime Text 3\\Packages\\User\\Preferences.sublime-settings"
                        }
                    }
                ]
            }
        },
        processes: []
    },
    {
        name: "Acceptance test for large font size in Sublime Text",
        userToken: "sublime_gert",
        settingsHandlers: {
            "gpii.settingsHandlers.JSONSettingsHandler": {
                "data": [
                    {
                        "settings": {
                            "font_size": 32
                        },
                        "options": {
                            "filename": "${{environment}.APPDATA}\\Sublime Text 3\\Packages\\User\\Preferences.sublime-settings"
                        }
                    }
                ]
            }
        },
        processes: []
    }
];

module.exports = gpii.test.bootstrap({
    testDefs:  "gpii.tests.windows.sublimetext",
    configName: "gpii.tests.acceptance.windows.sublimetext.config",
    configPath: "%universal/tests/platform/windows/configs"
}, ["gpii.test.integration.testCaseHolder.windows"],
    module, require, __dirname);

Below are a few comments to help you understand the above code.

• The line fluid.registerNamespace("gpii.tests.windows"); is specific to Windows. Acceptance tests for solutions on GNU/Linux will contain the string linux in the namespace, tests for Android solutions will contain the string android in the namespace, ectetera.
• The identifier gpii.tests.windows.sublimetext below the namespace registration code is referenced in the name-value pair testDefs: "gpii.tests.windows.sublimetext" near the end of the file.
• Each test has four parts: name, userToken, settingsHandlers and processes.
  • The key name provides a human-readable description of the test.
  • The key userToken refers to the NP set involved in the test.
  • The settingsHandlers block identifies the type of setting handler that is required for reading and writing the solution's settings. The value provided here, gpii.settingsHandlers.JSONSettingsHandler in the case of Sublime Text, matches the type of settings handler identified in the solution registry entry.
    The data block contains two parts:
    • settings contains the expected application-specific terms and their corresponding values after the NP set identified above has been gone through the transformers defined in the solutions registry entry.
    • options defines where the solution's settings can be found. This matches the options for the settings handler defined in the solutions registry entry.
      Note that options does not always use a file name reference to identify where settings are stored. See for example the acceptance tests for Android's built-in features, where the settings persistence system is defined as "settingType": "System".
  • The processes allows for the specification of an array of running applications to check. For Sublime Text, no such processes are relevant. For documentation on how to define processes, see the relevant section of the wiki page Writing Acceptance Tests.

In addition to the acceptance test file itself, you should also create a plain text file that describes the resources required for the tests (NP sets, etcetera).

Testing the Integration

(todo)

When the integration tests are complete, you should check that all the code in the pull request meets the criteria in the Pull Request Review Checklist.

Integration on GNU/Linux

Sublime Text has a version for GNU/Linux, but this version is not available through a package repository. Sublime Text can be installed in a non-standard way (for example, using the instructions in the article Installing Sublime Text 3 on RHEL Fedora), but will not be known to PackageKit. As a consequence, GPII, which currently relies on PackageKit to find installed applications, will not be able to find it. For this reason, the GPII integration of Sublime Text on GNU/Linux is not part of this tutorial.