Custom backgrounds in Microsoft Teams video calls

Microsoft Teams allows you to blur or replace the background with pre-defined images. However, there is no user interface which allows to upload custom images.

It turns out that it is just the upload button which is missing – if you know where the folder lies from which Teams fetches the backgrounds, you can easily place your own content in there.

  • On Windows, this is: %AppData%\Microsoft\Teams\Backgrounds\Uploads, i.e. C:\Users\<your user name>\AppData\Roaming\Microsoft\Teams\Backgrounds\Uploads
  • On Mac, this is: /users/<your user name>/Library/Application Support/Microsoft/Teams/Backgrounds/Uploads

This allows for some creative effects:

To subtle? What about this:

P.S.: One nifty trick to bewilder your colleagues: Take a screenshot during the next video call, use Gimp to retouch the person from the image and use their room as your background

(extra blur added for this screenshot)

How we organize a small development team with minimal overhead in Gitlab

At ioxp we develop a novel way of sharing knowledge using Augmented Reality, AI and Computer Vision to change the way how industries operate. Our ecosystem consists of many different tools and technologies – the AI backend, Hololens apps , Android apps, a web-based editor and cloud services – so we are very dependent on an efficient way to organize our developers and their different, often overlapping sub-teams.

And after all we are a startup and engineers by heart – we have to keep the balance between a thorough designed process and the creative freedom needed to explore and create new things.

This post will concentrate on how we facilitate developer communication and team work using features of the DevOps tool Gitlab, factoring out all other important stages like product management, planning or sprint planning. It is an excerpt from our internal Handbook (naturally hosted as Wiki in Gitlab) and describes the process I have designed and implemented together with my team.

The development process is primarily guided by issues and their three most important attributes: labels, assignees and related merge requests. It uses as little as 10 different issue labels and is divided into six stages:

Continue reading

How we keep an eye on our internal infrastructure with Gitlab – automated!

At ioxp, Gitlab is our tool of choice to organize our development. We also use their Wiki tool for our employee handbook and this is where we also have a list of all our internal services to maintain an overview about what is deployed, why and where. It looks like this:

To a) check that every service listed there is reachable and thus b) checking that the list is up-to-date (at least in one direction), we added a special Gitlab CI job which curl’s them all. This is its definition inside our main project’s .gitlab-ci.yml:

The script

Internal IT Check:
  only:
    variables:
      - $CI_CUSTOM_ITCHECK
  needs: []
  variables:
    GIT_STRATEGY: none
    WIKI_PAGE: IT-Address-List.md
  before_script:
    - apk add --update --no-cache git curl
    - git --version
    - git clone https://gitlab-ci-token:${CI_JOB_TOKEN}@${CI_SERVER_HOST}/wiki/wiki.wiki.git
  script:
    # Scrape the URLs from the IT wiki page
    - cd wiki.wiki
    - |
      urls="$(cat $WIKI_PAGE | grep -Ev '~~.+~~' | grep -Eo '(http|https)://([-a-zA-Z0-9@:%_+.~#?&amp;/=]+)' | sort -u)"
      if [ -z "$urls" ]; then echo "No URLs found in Wiki"; exit 1; fi
    - echo "Testing URLs from the Wiki:" &amp;amp;&amp;amp; echo "$urls"
    # Call each URL and see if it is available. set +e is needed to not exit on subshell errors. set -e is to revert to old behavior.
    # Special case: As one of our services does not like curls, we have to set a user agent
    - set +e
    - failures=""
    - for i in $urls ; do result="$(curl -k -sSf -H 'User-Agent:Mozilla' $i 2>&amp;amp;1 | cut -d$'\n' -f 1)"; if [ $? -ne 0 ]; then failures="$failures\n$i - $result"; fi; done
    - set -e
    - if [ -z "$failures" ] ; then exit 0; fi
    # Errors happened
    - |
      errorText="ERROR: The following services listed in the Wiki are unreachable. Either they are down or the wiki page $WIKI_PAGE is out-dated:"
      echo -e "$errorText\033[0;31m$failures\033[0m"
      curl -X POST -H "Content-Type: application/json" --data "{ 'pretext': '${CI_JOB_NAME} failed: ${CI_JOB_URL}', 'text': '$errorText$failures', 'color': 'danger' }" ${CI_CUSTOM_SLACK_WEBHOOK_IT} 2>&amp;amp;1
      exit 1

Script Breakup

  1. Gitlab wikis are a git repository under the hood. You can clone them via <projectUrl>.wiki.git (The URL can also be retrieved by clicking on “Clone repository” at the upper right on the wiki main page
  2. The URLs are extracted by grepping the respective markdown page for http/https links: grep -Eo '(http|https)://([-a-zA-Z0-9@:%_+.~#?&/=]+)'
  3. To ignore deprecated links (strikethrough in the wiki), we exclude them before with grep -Ev '~~.+~~' and to filter out duplicates we use sort -u afterwards
  4. Each found URL is then called via curl:
    • -k disables the TLS certificate verification if you use self-signed certificates and haven’t installed your own root CA.
    • -H 'User-Agent:Mozilla' was needed because one of our services didn’t like to be curl’ed by a non-browser so we dress up as one.
  5. With set +e we prevent that a subshell error (i.e. the curl) makes the Gitlab job exit with a failure prematurely as we want a list of all errors, not just the first one.
  6. If any error is found, a post containing the list of unavailable services is sent to our IT Slack channel using their webhooks.

Gitlab Preconditions

To run this job in a fixed interval, we set up a scheduled pipeline, running every day at two in the afternoon. The schedule has the variable CI_CUSTOM_ITCHECK defined so the job’s only: selector fires (see in the script above). All other jobs of our pipeline are either defined to not run under schedules or not when the variable is defined (using the except: selector).

Gitlab pipelines only/except for WIP merge requests

At ioxp we automate our continuous integration using the awesome Gitlab project. However, their model of when testing jobs run on commits induced quite a heavy load on our testing servers. So we were very delighted when pipelines for merge requests were introduced in Gitlab 11.6.

We then configured our testing to run with

only:merge_requests

However, our development process (more on that in a later post) recommends:

If a developer starts working on an issue, they have to

1. (if not already done) assign them
2. Create a branch starting with their two-letter initials, like ph/purposeOfTheBranch,
3. Open a new WIP merge request mentioning the issue to mark that it is currently being worked on,
4. Assign themselves to the merge request because they are currently responsible for it

We do this to have a nice overview about what is currently going on and to discuss about unfinished work. However, our only:merge_requests became useless now – as every implementation starts with a merge request, the pipelines are immediately started wasting CI time.

Luckily, variables expressions came to our rescue:

integrationtest:
  stage: integrationtest
  tags:
    - withBackend
  only:
    - merge_requests
  except:
    variables:
      - $CI_MERGE_REQUEST_TITLE =~ /^WIP:.*/

It is as easy as that.

Worry-free text writing into OpenCV images

Using cv::putText is cumbersome and placing your text at the correct position with the correct size is hard. Here is a wrapper function dealing with all of this for you. The text is fitted inside the given image, even multiple lines are possible and everything is nicely centered.

void ioxp::putText(cv::Mat imgROI, const std::string &text, const int fontFace = cv::FONT_HERSHEY_PLAIN,
    const cv::Scalar color = cv::Scalar::all(255), const int thickness = 1, const int lineType = cv::LINE_8)
{
    /*
     * Split the given text into its lines
     */
    std::vector<std::string> textLines;
    std::istringstream f(text);
    std::string s;
    while (std::getline(f, s, '\n')) {
        textLines.push_back(s);
    }

    /*
     * Calculate the line sizes and overall bounding box
     */
    std::vector<cv::Size> textLineSizes;
    cv::Size boundingBox(0,0);
    int baseline = 0;
    for (std::string line : textLines) {
        cv::Size lineSize = cv::getTextSize(line, fontFace, 1, thickness, &baseline);
        baseline += 2 * thickness;
        lineSize.width += 2 * thickness;
        lineSize.height += baseline;
        textLineSizes.push_back(lineSize);
        boundingBox.width = std::max(boundingBox.width, lineSize.width);
        boundingBox.height += lineSize.height;
    }

    const double scale = std::min(imgROI.rows / static_cast<double>(boundingBox.height),
                                  imgROI.cols / static_cast<double>(boundingBox.width));
    boundingBox.width *= scale;
    boundingBox.height *= scale;
    baseline *= scale;
    for (size_t i = 0; i < textLineSizes.size(); i++) {
        textLineSizes.at(i).width *= scale;
        textLineSizes.at(i).height *= scale;
    }
    /*
     * Draw the text line-by-line
     */
    int y = (imgROI.rows - boundingBox.height + baseline) / 2;
    for (size_t i = 0; i < textLines.size(); i++) {
        y += textLineSizes.at(i).height;
        // center the text horizontally
        cv::Point textOrg((imgROI.cols - textLineSizes.at(i).width) / 2, y - baseline);
        cv::putText(imgROI, textLines.at(i), textOrg, fontFace, scale, color, thickness, lineType);
    }
}

This is how you use it and how the results look like:

    cv::Mat outputImage(360, 640, CV_8UC3);
    outputImage.setTo(0);
    ioxp::putText(outputImage, "Short text");
    cv::imshow("text", outputImage);
    cv::waitKey(0);

    cv::Mat outputImage(360, 640, CV_8UC3);
    outputImage.setTo(0);
    ioxp::putText(outputImage,
      "Some longer text, even with\nmultiple lines spread over the whole image");
    cv::imshow("text", outputImage);
    cv::waitKey(0);

    cv::Mat outputImage(360, 640, CV_8UC3);
    outputImage.setTo(0);
    ioxp::putText(outputImage, "\n\n\nEmpty\n\n\nLines\n\n\n");
    cv::imshow("text", outputImage);
    cv::waitKey(0);

By using the Rectangle accessor you can define exactly, which part of the image the text should be placed in:

    cv::Mat outputImage(360, 640, CV_8UC3);
    outputImage.setTo(0);
    ioxp::putText(outputImage(cv::Rect(0, outputImage.rows / 10, outputImage.cols, outputImage.rows / 10)),
      "Text placed in the upper\n10 percent of the image");
    cv::imshow("text", outputImage);
    cv::waitKey(0);

Combining ARCore tracking and Cardboard Spatial Audio

This week Google released ARCore, their answer to Apple’s recently published Augmented Reality framework ARKit. This is an exciting opportunity for mobile developers to enter the world of Augmented Reality, Mixed Reality, Holographic Games, … whichever buzzword you prefer.

To get to know the AR framework I wanted to test how easy it would be to combine it with another awesome Android framework: Google VR, used for their Daydream and Cardboard platform. Specifically, its Spatial Audio API. And despite never having used one of those two libraries, combining them is astonishingly simple.

Cf. to https://developers.google.com/vr/concepts/spatial-audio

The results:

The goal is to add correctly rendered three dimensional sound to an augmented reality application. For a demonstrator, we pin an audio source to each of the little Androids placed in the scene.
Well, screenshots don’t make sense to demonstrate audio but without them this post looks so lifeless 🙂 Unfortunately, I could not manage to do a screen recording which includes the audio feed.

The how-to:

  1. Setup ARCore as explained in the documentation. Currently, only Google Pixel and the Samsung Galaxy S8 are supported so you need one of those to test it out. The device coverage will increase in the future
  2. The following step-by-step tutorial starts at the sample project located in /samples/java_arcore_hello_ar it is based on the current Github repository’s HEAD
  3. Open the application’s Gradle build file at /samples/java_arcore_hello_ar/app/build.gradle and add the VR library to the dependencies
    dependencies {
        ...
        compile 'com.google.vr:sdk-audio:1.10.0'
    }
    
  4. Place a sound file in the asset folder. I had some troubles getting it to work until I found out that it has to be a 32-bit float mono wav file. I used Audacity for the conversion:
    1. Open your Audio file in Audacity
    2. Click Tracks -> Stereo Track to Mono
    3. Click File -> Export. Select “Other uncompressed files” as type, Click Options and select “WAV” as Header and “Signed 32 bit PCM” as encoding

    I used “Sam’s Song” from the Ubuntu Touch Sound Package and you can download the correctly converted file here.

  5. We have to apply three modifications to the sample’s HelloArActivity.java: (1) bind the GvrAudioEngine to the Activity’s lifecycle, (2) add a sound object for every object placed into the scene and (3) Continuously update audio object positions and listener position. You find the relevant sections below.

    public class HelloArActivity extends AppCompatActivity implements GLSurfaceView.Renderer {
        /*
        ...
        */
        private GvrAudioEngine mGvrAudioEngine;
        private ArrayList&lt;Integer&gt; mSounds = new ArrayList&lt;&gt;();
        final String SOUND_FILE = "sams_song.wav";
    
        @Override
        protected void onCreate(Bundle savedInstanceState) {
            /*
            ...
             */
            mGvrAudioEngine = new GvrAudioEngine(this, GvrAudioEngine.RenderingMode.BINAURAL_HIGH_QUALITY);
            new Thread(
                new Runnable() {
                    @Override
                    public void run() {
                        // Prepare the audio file and set the room configuration to an office-like setting
                        // Cf. https://developers.google.com/vr/android/reference/com/google/vr/sdk/audio/GvrAudioEngine
                        mGvrAudioEngine.preloadSoundFile(SOUND_FILE);
                        mGvrAudioEngine.setRoomProperties(15, 15, 15, PLASTER_SMOOTH, PLASTER_SMOOTH, CURTAIN_HEAVY);
                    }
                })
            .start();
        }
    
        @Override
        protected void onResume() {
            /*
            ...
             */
            mGvrAudioEngine.resume();
        }
    
        @Override
        public void onPause() {
            /*
            ...
             */
            mGvrAudioEngine.pause();
        }
    
        @Override
        public void onDrawFrame(GL10 gl) {
            // Clear screen to notify driver it should not load any pixels from previous frame.
            GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
    
            try {
                // Obtain the current frame from ARSession. When the configuration is set to
                // UpdateMode.BLOCKING (it is by default), this will throttle the rendering to the
                // camera framerate.
                Frame frame = mSession.update();
    
                // Handle taps. Handling only one tap per frame, as taps are usually low frequency
                // compared to frame rate.
                MotionEvent tap = mQueuedSingleTaps.poll();
                if (tap != null &amp;&amp; frame.getTrackingState() == TrackingState.TRACKING) {
                    for (HitResult hit : frame.hitTest(tap)) {
                        // Check if any plane was hit, and if it was hit inside the plane polygon.
                        if (hit instanceof PlaneHitResult &amp;&amp; ((PlaneHitResult) hit).isHitInPolygon()) {
                            /*
                            ...
                             */
                            int soundId = mGvrAudioEngine.createSoundObject(SOUND_FILE);
                            float[] translation = new float[3];
                            hit.getHitPose().getTranslation(translation, 0);
                            mGvrAudioEngine.setSoundObjectPosition(soundId, translation[0], translation[1], translation[2]);
                            mGvrAudioEngine.playSound(soundId, true /* looped playback */);
                            // Set a logarithmic rolloffm model and mute after four meters to limit audio chaos
                            mGvrAudioEngine.setSoundObjectDistanceRolloffModel(soundId, GvrAudioEngine.DistanceRolloffModel.LOGARITHMIC, 0, 4);
                            mSounds.add(soundId);
    
                            // Hits are sorted by depth. Consider only closest hit on a plane.
                            break;
                        }
                    }
                }
                /*
                 ...
                 */
                // Visualize planes.
                mPlaneRenderer.drawPlanes(mSession.getAllPlanes(), frame.getPose(), projmtx);
    
                // Visualize anchors created by touch.
                float scaleFactor = 1.0f;
                for (int i=0; i &lt; mTouches.size(); i++) {
                    PlaneAttachment planeAttachment = mTouches.get(i);
                    if (!planeAttachment.isTracking()) {
                        continue;
                    }
                    // Get the current combined pose of an Anchor and Plane in world space. The Anchor
                    // and Plane poses are updated during calls to session.update() as ARCore refines
                    // its estimate of the world.
                    planeAttachment.getPose().toMatrix(mAnchorMatrix, 0);
    
                    // Update and draw the model and its shadow.
                    mVirtualObject.updateModelMatrix(mAnchorMatrix, scaleFactor);
                    mVirtualObjectShadow.updateModelMatrix(mAnchorMatrix, scaleFactor);
                    mVirtualObject.draw(viewmtx, projmtx, lightIntensity);
                    mVirtualObjectShadow.draw(viewmtx, projmtx, lightIntensity);
    
                    // Update the audio source position since the anchor might have been refined
                    float[] translation = new float[3];
                    planeAttachment.getPose().getTranslation(translation, 0);
                    mGvrAudioEngine.setSoundObjectPosition(mSounds.get(i), translation[0], translation[1], translation[2]);
                }
    
                /*
                 * Update the listener's position in the audio world
                 */
                // Extract positional data
                float[] translation = new float[3];
                frame.getPose().getTranslation(translation, 0);
                float[] rotation = new float[4];
                frame.getPose().getRotationQuaternion(rotation, 0);
    
                // Update audio engine
                mGvrAudioEngine.setHeadPosition(translation[0], translation[1], translation[2]);
                mGvrAudioEngine.setHeadRotation(rotation[0], rotation[1], rotation[2], rotation[3]);
                mGvrAudioEngine.update();
            } catch (Throwable t) {
                // Avoid crashing the application due to unhandled exceptions.
                Log.e(TAG, "Exception on the OpenGL thread", t);
            }
        }
        /*
         ...
         */
    }
    
    
  6. That’s it! Now, every Android placed into the scene also plays back audio.

Some findings:

  1. Setting up ADB via WiFi is really helpful as you will walk around a lot and don’t want to reconnect USB every time.
  2. Placing the Androids too close to each other will produce a really annoying sound chaos. You can modify the rolloff model to reduce this (cf. line 71 in the code excerpt above).
  3. It matters how you hold your phone (portrait with the current code), because ARCore measures the physical orientation of the device but the audio coordinate system is (not yet) rotated accordingly. If you want to use landscape mode, it is sufficient to set the Activity in the manifest to android:screenOrientation="landscape"
  4. Ask questions tagged with the official arcore tag on Stack Overflow, the Google developers are reading them!

War stories: When the visual debugger fails you

I recently had a very strange crash and after some digging I found the lines I suspected the bug to lurk around. They looked something like this:

    const std::string contents = readFile("myFile.txt");
    const std::vector<std::string> lines = utils::split(contents, "\n");
    for (std::string line : lines) {
        if (line.empty()) {
            continue;
        }
        //...
        // Do something elaborate with the line, e.g. printing to console
        std::cout << "<line>" << line.c_str() << "</line>" << std::endl;
    }

The crash occurred in the //... lines because the line was not empty. Wait – what? I tested for emptiness before!
Opening the debugger revealed the following strange situation:

and the above small sample file prints on my (Windows) console:

<line>First line</line>
<line>third line (second one is empty)</line>
<line>fourth line</line>
<line></line>

Scrolling trough the commit history, the problem turned out to be introduced with this change:
OLD CODE (working):

std::string readFile(const std::string& fname, bool binaryMode = false)
{
    std::ios_base::openmode mode = std::ios_base::in;
    if (binaryMode)
        mode |= std::ios_base::binary;

    std::ifstream ifs(fname, mode);
    if (!ifs) {
        throw std::exception(("Couldn't open file: " + fname).c_str());
    }
    return std::string(std::istreambuf_iterator<char>(ifs), std::istreambuf_iterator<char>());
}

NEW CODE (not working):

std::string readFile(const std::string& fname, bool binaryMode = false)
{
    std::ios_base::openmode mode = std::ios_base::in;
    if (binaryMode) {
        mode |= std::ios_base::binary;
    }

    std::ifstream in(fname, mode);
    if (!in) {
        throw std::exception(("Couldn't open file: " + fname).c_str());
    }

    // Used C++ style reading which is more efficient than using stream buffer iterators
    // http://insanecoding.blogspot.de/2011/11/how-to-read-in-file-in-c.html
    std::string contents;
    in.seekg(0, std::ios::end);
    contents.resize(in.tellg());
    in.seekg(0, std::ios::beg);
    in.read(&contents[0], contents.size());
    in.close();

    return contents;
}

The problem was that reading the file with the more efficient solution resulted in the string having a bunch of null terminators if the file contained a new-line in the end. Obviously, .empty() returns false, so the check passed. As a side note: to simulate my crash bug by showing an unexpected console output, I had to pipe the C-string. When piping the C++ string, the line is printed with some whitespaces.

This is how I fixed it:

std::string readFile(const std::string& fname, bool binaryMode = false)
{
    std::ios_base::openmode mode = std::ios_base::in;
    if (binaryMode) {
        mode |= std::ios_base::binary;
    }

    std::ifstream in(fname, mode);
    if (!in) {
        throw std::exception(("Couldn't open file: " + fname).c_str());
    }

    // Used C++ style reading which is more efficient than using stream buffer iterators
    // http://insanecoding.blogspot.de/2011/11/how-to-read-in-file-in-c.html
    std::string contents;
    in.seekg(0, std::ios::end);
    contents.resize(in.tellg());
    in.seekg(0, std::ios::beg);
    in.read(&contents[0], contents.size());
    in.close();

    if (binaryMode) {
        return contents;
    }
    else {
        // Depending on the file, the last line might contain one or more \0 control characters. Remove them
        return contents.erase(contents.find_last_not_of('\0') + 1);
    }
}

Git stop words

If you add development code lines to your file which must not be committed to the code base (e.g. temporarily disabled code, fixated variables, noisy outputs), mark them with the word NOCOMMIT by putting it in a comment, a variable name, …

if(true) {//NOCOMMIT counter > 5 && testThis)
    myNewFeatureToTest();
} else {
    someOtherCode();
}
int NOCOMMIT = 5;
int myVariable = 1;
myVariable = 5; //NOCOMMIT

To activate instant rejection by git as soon as you try to commit this code, do the following:

1. Put this helper script in the file your repository checkout/.git/hooks/showlinenum.awk
2. and the following hook in the file your repository checkout/.git/hooks/pre-commit:

#!/bin/sh
#
# Dismisses the commit if it adds illegal statements (see variable $searchPhrase)

if git rev-parse --verify HEAD >/dev/null 2>&1
then
    against=HEAD
else
    # Initial commit: diff against an empty tree object
    against=4b825dc642cb6eb9a060e54bf8d69288fbee4904
fi

# Redirect output to stderr.
exec 1>&2
# ""
# Test for NOCOMMIT and other typical debugging lines
searchPhrase='NOCOMMIT\|(true ||\|(true||\|(false &&\|(false&&'

DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )

. git-sh-setup  # for die
git-diff-index -p -M --cached $against -- | $DIR/showlinenum.awk show_path=1 | grep -E '^.+[0-9]+:\+' |
grep "$searchPhrase" && echo "" && die Rejected commit since the above lines contain illegal statements. Use git commit -n to ignore

# If there are whitespace errors, print the offending file names and fail.
exec git diff-index --check --cached $against --

Tip1: best add the stop word to a line which will cause a compile failure if you would commit the lines around it but forgot the one with the stop word (cf. the if condition above).

Tip2: if you want to ignore this hook for one commit, use git commit -n -m "[tag] Your commit message".

Indicate the first run after new push from Android Studio

When developing an app with a big file size the time between pressing Ctrl+F5 and having the new app instance running on a connected device can be rather long.

A timespan of around 10 seconds for a 25MB app is long enough for me to deal with other things, e.g. staging my changes in the VCS and then looking back at the device wondering whether the new version is already running or if I’m looking at the old state. Usually I then start clicking to test the new implementation when the app just closes and reopens since the upload took longer than expected.

The solution is: Add a script to AndroidStudio’s build process which closes the app immediately after pressing Ctrl+F5. This way, when you see your app screen the next time, you can be sure that you are looking at the new version.

  1. Open Android Studio → Run → Edit Configurations
  2. Select your application, scroll to the bottom to the “Before launch” section. Click Plus -> Run External Tool -> Click Plus.Set the values:
    Name: Force-stop app
    program: adb.exe
    parameters: shell am force-stopcreateTool 
  3. Make sure that the external tool runs before the Gradle-aware Make in the “Before Launch” sectionrunDebugConfigurations

P.S.: An alternative would be to auto-increment the version code with every change but that is not feasible for me since I increment my version code automatically based on my git history (more on that later).

Microsoft Surface: remapping keys for power users

The Surface Pro 3 is a neat device but there are a bunch of user experience flaws concerning the Cover and the Pen (which we will solve in this post) such as:

  1. The Cover’s function keys F1-F12 can only be reached using the Fn button – especially annoying for developers.
  2. The Cover has no button for changing the screen brightness.
  3. The Surface Pen’s top button (the purple one) cannot be configured to open a custom program. I want the Snipping Tool to be opened.
  4. The Surface Pen lacks a “back” button, e.g. for quickly correct a wrong pen stroke in Photoshop, Mischief or similar drawing programs.
  5. I never use CapsLock – let’s map it to a different key, e.g. AltGr (this is of course not Surface-specific but a nice productivity boost nonetheless)

Solutions:

The solutions 1 & 2 are just simple keyboard shortcuts:

  1. Function key lock: Press Fn+CapsLock to toggle the behavior. Unfortunately, this introduces a new problem which will be solved in the paragraph after this:
    1. For F1-F8 I want the default to be the function keys themselves but for F9-F12 I want the reverse as default: Home, End, Page Up, Page DownSurface Pro cover: function keys
  2. Secret shortcuts for screen brightness: Fn + Del/Backspace

The remaining problems are solved by using AutoHotKey with its extensive scripting capabilities and a lively community sharing knowledge thereof.
After installing it, copy the following script to the autostart folder (e.g. under the name of EnableSurfaceProductivity.ahk) to have it running with every start. To test the effect immediately, double-click the file.

; Solution for 1b: Reverse behavior for F9-F12
; The dollar signs prevent that the hooks call themselves

$Home::SendInput, {F9}
return

$End::SendInput, {F10}
return

$PgUp::SendInput, {F11}
return

$PgDn::SendInput, {F12}
return

$F9::Send, {Home}
return

$F10::Send, {End}
return

$F11::Send, {PgUp}
return

$F12::Send, {PgDn}
return

; Solution 3: Open the Snipping Tool when double-pressing the purple button
; The script waits for the program to start and then simulates
; Win+PrintScreen which directly jumps to the selection process
#F19::
Run, "C:\Windows\Sysnative\SnippingTool.exe"
WinActivate, Snipping Tool ; This makes sure it is active
WinWaitActive, Snipping Tool
Send, ^{PrintScreen}
return

; Solution 4: Simulate Ctrl+Z when single-clicking the purple button
#F20::Send, ^z ; Left Arrow, Browser Back Button
return

; Solution 5: Map CapsLock to AltGr
CapsLock::RAlt