Control Computer from mobile using Node JS

Aniket Bhattacharyea -

I am a lazy f***, and sometimes, while working on my laptop, I don’t feel like stretching my hand aaaaaallllll the way to the laptop. I use an app on my phone called KDE Connect. This is a pretty powerful app. It has a counterpart for the desktop and the android phone, and through it, you can control any device running KDE Connect in the same network. You can send files, run commands, control keyboard and mouse, control media player, sync notifications, share clipboard and what not! Do check it out!!

Today I want to try if I can write a similar software or not. Of course this is not going to be as feature rich as KDE Connect. I will just create a server which will run on my lappy, and through the web browser in my phone, I will be able to talk to the server – send keystrokes and control mouse.

First, let’s start with the easy one – sending keystrokes. Here’s the complete flow –

  1. The server listens on a port on the lappy.
  2. My phone connects to the server and asks for pairing.
  3. I authorize the pairing on the laptop. This is to prevent any random device in my network to send keystrokes.
  4. Server will establish a session with my phone. This is to prevent any other authorized device to randomly stroke my laptop in between. The server will check every 30 seconds for a ping from my phone. If there’s no ping in 30 seconds, it will assume the session is closed.
  5. My phone will send a keystroke, and the server will capture it, and emulate pressing the key on the laptop.

Let’s start!

For the server, I’ll use Node JS. And to emulate the keypress, I’ll use this package.

Setting Up

First create the directory and move into it –

mkdir NodeServer
cd NodeServer

And initialize npm –

npm init

Press enter through all the questions.

Next, install the required packages –

npm install express node-key-sender --save

Write the code

Create a file called index.js, and write the below code –

const express = require('express');
const app = express();
const server = require('http').createServer(app);
const router = express.Router();
app.use(express.json());
app.use(express.urlencoded({
  extended: false
}));

app.use('/', router);

router.get('/ping', (req, res) => {
  res.send('pong');
});


server.listen(3000, () => {
  console.log("Server up!!");
});

Save it, and run it by

node index.js

You should see a message server up!!. Let’s test it out. Open another terminal and run this command –

curl localhost:3000/ping

You should get a response pong

Now, let’s go to the real coding!

We will have four routes –

  1. /pair – to pair the devices
  2. /start – to start the session
  3. /key – to send the keystroke
  4. /ping – to send a ping every 30 seconds to keep the connection alive.

Let’s start with the first one. A post request to /pair will produce a token and send it to the client. The client must include the token in subsequent requests. The token will be stored in the server to check for authentication. When a request comes, the server will show a notification to the user and according to the users choice, will accept or reject the notification.

Here’s a catch. For sending notifications, I will be using the freedesktop-notifications library on Linux. If you’re using Windows or Mac, you might want to change this part.

First we’ll install the library –

npm install --save freedesktop-notifications

Then in index.js, we’ll require the libraries –

const crypto = require('crypto');
const notification = require('freedesktop-notifications');

crypto will be used to generate a random token.

Then we need to keep a track of tokens. So we create an array of generated tokens –

const authorized_tokens = [];

And then add a new route for /pair –

router.post('/pair', (req, res) => {
  let handled = false;
  let notif = notification.createNotification({
    summary: 'Pairing request',
    body: req.body.name + " wants to pair",
    actions: {
      ok: 'Accept',
      nope: 'Reject'
    }
  });

  notif.on('action', (action) => {
    if(action === 'default' || action === 'nope') {
      handled = true;
      return res.json({
        token: null,
        rejected: true
      });
    }
    let token;
    do {
      token = crypto.randomBytes(16).toString('hex');
    } while(authorized_tokens.includes(token));
    authorized_tokens.push(token);
    handled = true;
    return res.json({
      token: token,
      rejected: false
    });

  });
  notif.on('close', (closed_by) => {
    if(closed_by === 'timeout' || (closed_by === 'user' && !handled))
    return res.json({
      token: null,
      rejected: true
    });
  });
  notif.push();
});

Let’s break it down!

We are first creating a boolean handled to keep track of if the notification was handled or not. The reason will be clear soon.

Then we are creating the notification and using the name parameter from the request body to show the notification. There are 2 (actually 3) actions –

  1. ok: We are naming it accept
  2. nope: We are naming it reject
  3. default: This will be emitted when the user clicks the notification itself. We will count this as rejection, because we are a-holes.

The action event will be emitted if the user performs any of the three actions. If the user clicks the reject button or the notification itself, we are sending a response to the client letting them know about the rejection. We are also setting handled = true.

If the user clicks accept, we are generating a random 16 bytes hex string, and checking if it’s present already in the array or not. Although the probability is pretty low, still, I am a mathematician, and “pretty low” for me doesn’t mean “never going to happen.” So we keep trying until we find a unique token. We put that in the array and send the same to the client. Also handled = true

Now the closed event is little wonky. If the user doesn’t click anything, but lets the notification expire, we’ll count that as rejection too, because we are a-holes.

The close event will be emitted even if the notification is closed as a result of user clicking the button. This is where handled comes in. If the notification is closed by timeout, or the user but handled is false (which means user cancelled the notification), we got a rejection!!

Finally we are pushing the notification.

Let’s try.

Run the server as before, and run this command –

curl -X POST localhost:3000/pair -d “name=aniket”

You should see a notification

Howdy Ho!

If you click the accept button, you should see a response –

{"token":"d88f9175a422f137f12299d3cba86311","rejected":false}%

And if you click the reject button, or the close button, or let it expire, you’ll see a response –

{"token":null,"rejected":true}

Yay! That’s one down!

Here’s the whole of index.js at this point –

const express = require('express');
const crypto = require('crypto');
const notification = require('freedesktop-notifications');
const app = express();
const server = require('http').createServer(app);
const router = express.Router();
app.use(express.json());
app.use(express.urlencoded({
  extended: false
}));

const authorized_tokens = [];
app.use('/', router);

router.get('/ping', (req, res) => {
  res.send('pong');
});

router.post('/pair', (req, res) => {
  let handled = false;
  let notif = notification.createNotification({
    summary: 'Pairing request',
    body: req.body.name + " wants to pair",
    actions: {
      ok: 'Accept',
      nope: 'Reject'
    }
  });

  notif.on('action', (action) => {
    if(action === 'default' || action === 'nope') {
      handled = true;
      return res.json({
        token: null,
        rejected: true
      });
    }
    let token;
    do {
      token = crypto.randomBytes(16).toString('hex');
    } while(authorized_tokens.includes(token));
    authorized_tokens.push(token);
    handled = true;
    return res.json({
      token: token,
      rejected: false
    });

  });
  notif.on('close', (closed_by) => {
    if(closed_by === 'timeout' || (closed_by === 'user' && !handled))
    return res.json({
      token: null,
      rejected: true
    });
  });
  notif.push();
});
server.listen(3000, () => {
  console.log("Server up!!");
});

The next route – /start is pretty easy! The client just needs to send the token to the server and a session will be established. There are three things to check –

  1. Is a session already running? If yes, reject.
  2. Is the sent token valid? If not reject?
  3. We also need to store the token which currently is holding a session.

We create two variables –

let in_session = false;
let current_session;

And here’s the code for the /start route –

router.post('/start', (req, res) => {
  if(in_session) return res.json({
    success: null,
    error: 'Already in a session'
  });
  if(!authorized_tokens.includes(req.body.token)) return res.json({
    success: null,
    error: 'Not authorized'
  });
  in_session = true;
  current_session = req.body.token;
  res.json({
    success: 'Session established',
    error: null
  });
});

This code is pretty obvious. We are just performing the 3 steps.

Let’s test it! First start the server and get a token by accepting the notification as stated earlier and copy the token.

This is the command we will use –

curl -X POST localhost:3000/start -d "token=your_token_here"

First try with an unauthorized token. You should get a response –

{"success":null,"error":"Not authorized"}

Then try with the valid token –

{"success":"Session established","error":null}

And now if you try again –

{"success":null,"error":"Already in a session"}

Beautiful! Now let’s move to the ping route. But first, we haven’t added the timeout in session yet. So let’s do that.

First we will create a timer object –

let timer;

Then we modify the /start route by adding the timeout after we establish a session –

  ...
  in_session = true;
  current_session = req.body.token;
  timer = setTimeout(() => {
    console.log("Session with "+ current_session + " closed");
    in_session = false;
    current_session = '';
  }, 30*1000);
  res.json({
     ...

This starts a timer to execute 30 seconds from the time we start the session. After 30 seconds, we close the session by setting in_session = false and clearing current_session value.

Let’s try. First establish a session as before, and wait for 30 seconds and you should see a message like –

Session with de179f5044c2decea5b583945df8b7d1 closed

And you’ll be able to start a session again!

Now, the ping route. This will be easy too. We will –

  1. Check if a session is running or not.
  2. Check if the token parameter matches with the current_session value or not.
  3. If it matches, reset the timer.

Here’s the code –

router.post('/ping', (req, res) => {
  if(!in_session) return res.json({
    success: null,
    error: "No session running"
  });
  if(req.body.token !== current_session) return res.json({
    success: null,
    error: "Not in a session with the requested token"
  });
  clearTimeout(timer);
  in_session = true;
  current_session = req.body.token;
  timer = setTimeout(() => {
    console.log("Session with " + current_session + " closed");
    in_session = false;
    current_session = '';
  }, 30 * 1000);
  console.log("Session with " + current_session + " is extended");
  res.json({
    success: true,
    error: null
  });
});

This one’s pretty easy too. Once we know the token matches, we are clearing the timeout and resetting it.

You should be able to try that out by establishing a session and hitting the /ping route with your token.

Now let’s move to the real part!

The /key route will be simple too. We will –

  1. First check if the token matches with current_session or not.
  2. If it matches, we will take the key paramter, which should be a character. If it’s a string, we will take the first character because we are a-holes.
  3. Then we send the keystroke.

Here’s the code –

router.post('/key', (req,res) => {
  if(current_session !== req.body.token) return res.json({
    success: null,
    error: "Establish a session first"
  });

  const key = req.body.key[0];
  ks.sendLetter(key).then((stdout, stderr) => {
    res.json({
      success: true,
      error: null
    });
  }).catch((err, stdout, stderr) => {
    res.json({
      success: false,
      error: err
    });
  });
});

It’s pretty self explanatory. So I’m not wasting time explaining it. Let’s move to the testing!

One thing you should remember, the sendLetter method converts the letters into their corresponding keycode according to the keyboard layout. So if yur phone has some keys that your computer does not, you won’t be able to type that key. For example, I can’t type the British Pound symbol.

Quickly, pair, start session and run this command –

curl -X POST localhost:3000/key -d "token=f4767b860aaf1ca77695e9ef0dd0de52&key=A"

Replace the token with yours.

If you did correctly, you’ll see the A getting written in the terminal –

Hello, A!

See the ‘A’ at the beginning? That was echoed by the server!

Here’s the entire file –

const express = require('express');
const crypto = require('crypto');
const notification = require('freedesktop-notifications');
const ks = require('node-key-sender');
const app = express();
const server = require('http').createServer(app);
const router = express.Router();
app.use(express.json());
app.use(express.urlencoded({
  extended: false
}));

const authorized_tokens = [];
let timer;
let in_session = false;
let current_session;
app.use('/', router);

router.post('/ping', (req, res) => {
  if(!in_session) return res.json({
    success: null,
    error: "No session running"
  });
  if(req.body.token !== current_session) return res.json({
    success: null,
    error: "Not in a session with the requested token"
  });
  clearTimeout(timer);
  in_session = true;
  current_session = req.body.token;
  timer = setTimeout(() => {
    console.log("Session with " + current_session + " closed");
    in_session = false;
    current_session = '';
  }, 30 * 1000);
  console.log("Session with " + current_session + " is extended");
  res.json({
    success: true,
    error: null
  });
});

router.post('/pair', (req, res) => {
  let handled = false;
  let notif = notification.createNotification({
    summary: 'Pairing request',
    body: req.body.name + " wants to pair",
    actions: {
      ok: 'Accept',
      nope: 'Reject'
    }
  });

  notif.on('action', (action) => {
    if(action === 'default' || action === 'nope') {
      handled = true;
      return res.json({
        token: null,
        rejected: true
      });
    }
    let token;
    do {
      token = crypto.randomBytes(16).toString('hex');
    } while(authorized_tokens.includes(token));
    authorized_tokens.push(token);
    handled = true;
    return res.json({
      token: token,
      rejected: false
    });

  });
  notif.on('close', (closed_by) => {
    if(closed_by === 'timeout' || (closed_by === 'user' && !handled))
    return res.json({
      token: null,
      rejected: true
    });
  });
  notif.push();
});

router.post('/start', (req, res) => {
  if(in_session) return res.json({
    success: null,
    error: 'Already in a session'
  });
  if(!authorized_tokens.includes(req.body.token)) return res.json({
    success: null,
    error: 'Not authorized'
  });
  in_session = true;
  current_session = req.body.token;
  timer = setTimeout(() => {
    console.log("Session with "+ current_session + " closed");
    in_session = false;
    current_session = '';
  }, 30*1000);
  res.json({
    success: 'Session established',
    error: null
  });
});

router.post('/key', (req,res) => {
  if(current_session !== req.body.token) return res.json({
    success: null,
    error: "Establish a session first"
  });

  const key = req.body.key[0];
  ks.sendLetter(key).then((stdout, stderr) => {
    res.json({
      success: true,
      error: null
    });
  }).catch((err, stdout, stderr) => {
    res.json({
      success: false,
      error: err
    });
  });
});
server.listen(3000, () => {
  console.log("Server up!!");
});

Writing the interface

We are almost done, now we just need to write the HTML interface where we will be inputting from our phone.

It will be simple. The interface will have –

  1. A text input to put the name.
  2. A button to send the pairing request.
  3. A textbox to write the inputs

In the script side, we will –

  1. Send the pair request.
  2. If accepted, start a session.
  3. Send whatever the user types

Create a file called index.html and paste the following –

<!DOCTYPE html>
<html>
    <head>
        <script src="https://code.jquery.com/jquery-3.3.1.min.js"
            integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8="
            crossorigin="anonymous"></script>
    </head>
    <body>
        <label>Enter your name</label><br>
        <input type="text" id="name"><br>
        <button id="pair">Pair</button><br>
        <textarea id="text"></textarea><br>
        <script>
            let token;
            $('#pair').on('click', function(e){
                $.post('/pair', {
                    name: $('#name').val()
                }, function(result){
                    if(result.rejected) {
                        alert("Rejected");
                    }
                    else {
                        token = result.token;  
                        start_session();
                    }
                });
            });
            $('#text').on('input', function(e) {
                let key = $(this).val();
                $.post('/key',{
                    token: token,
                    key: key[key.length - 1]
                }, function(result) {
                    if(!result.success) {
                        alert("Error");
                        console.log(result.error);
                    }
                });
            });
            
            function start_session() {
                $.post('/start', {
                    token: token
                }, function(result) {
                    if(!result.success) {
                        alert("Error");
                        console.log(result.error);
                    }
                    else {
                        setInterval(function(){
                            $.post('/ping',{
                                token: token
                            }, function(result) {
                                if(!result.success) {
                                    alert("Error");
                                    console.log(result.error);
                                }
                            });
                        }, 20*1000);
                    }
                });
            }
        </script>
    </body>
</html>

I know, it’s ugly as f***, but in later parts of this guide (if I write one, that is), we’ll make it into an app 😉

We’re also using JQuery because we’re not barbarians. Also, we’re currently just logging in the errors. We’ll later handle them.

The code is easy. We pair, and then start a session, and ping every 20 seconds to keep it alive.

When the input event occurs on the textarea, we are taking its value and the last character would be our key which we send to the server.

Now we need a way to serve this file. We will serve this file from the /  route

router.get('/', (req, res) => res.sendFile('/home/aniket/NodeServer/index.html'));

Here's the full file in all its glory

const express = require('express');
const crypto = require('crypto');
const notification = require('freedesktop-notifications');
const ks = require('node-key-sender');
const app = express();
const server = require('http').createServer(app);
const router = express.Router();
app.use(express.json());
app.use(express.urlencoded({
  extended: false
}));

const authorized_tokens = [];
let timer;
let in_session = false;
let current_session;
app.use('/', router);

router.get('/', (req, res) => res.sendFile('/home/aniket/NodeServer/index.html'));

router.post('/ping', (req, res) => {
  if(!in_session) return res.json({
    success: null,
    error: "No session running"
  });
  if(req.body.token !== current_session) return res.json({
    success: null,
    error: "Not in a session with the requested token"
  });
  clearTimeout(timer);
  in_session = true;
  current_session = req.body.token;
  timer = setTimeout(() => {
    console.log("Session with " + current_session + " closed");
    in_session = false;
    current_session = '';
  }, 30 * 1000);
  console.log("Session with " + current_session + " is extended");
  res.json({
    success: true,
    error: null
  });
});

router.post('/pair', (req, res) => {
  let handled = false;
  let notif = notification.createNotification({
    summary: 'Pairing request',
    body: req.body.name + " wants to pair",
    actions: {
      ok: 'Accept',
      nope: 'Reject'
    }
  });

  notif.on('action', (action) => {
    if(action === 'default' || action === 'nope') {
      handled = true;
      return res.json({
        token: null,
        rejected: true
      });
    }
    let token;
    do {
      token = crypto.randomBytes(16).toString('hex');
    } while(authorized_tokens.includes(token));
    authorized_tokens.push(token);
    handled = true;
    return res.json({
      token: token,
      rejected: false
    });

  });
  notif.on('close', (closed_by) => {
    if(closed_by === 'timeout' || (closed_by === 'user' && !handled))
    return res.json({
      token: null,
      rejected: true
    });
  });
  notif.push();
});

router.post('/start', (req, res) => {
  if(in_session) return res.json({
    success: null,
    error: 'Already in a session'
  });
  if(!authorized_tokens.includes(req.body.token)) return res.json({
    success: null,
    error: 'Not authorized'
  });
  in_session = true;
  current_session = req.body.token;
  timer = setTimeout(() => {
    console.log("Session with "+ current_session + " closed");
    in_session = false;
    current_session = '';
  }, 30*1000);
  res.json({
    success: 'Session established',
    error: null
  });
});

router.post('/key', (req,res) => {
  if(current_session !== req.body.token) return res.json({
    success: null,
    error: "Establish a session first"
  });

  const key = req.body.key[0];
  ks.sendLetter(key).then((stdout, stderr) => {
    res.json({
      success: true,
      error: null
    });
  }).catch((err, stdout, stderr) => {
    res.json({
      success: false,
      error: err
    });
  });
});
server.listen(3000, () => {
  console.log("Server up!!");
});

Let’s test. First run this command to find out your computer’s internal IP address –

ip addr

You’ll get an output like this –

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp2s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 68:f7:28:eb:62:88 brd ff:ff:ff:ff:ff:ff
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether e4:f8:9c:10:2b:79 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.100/24 brd 192.168.0.255 scope global dynamic noprefixroute wlp3s0
       valid_lft 6398sec preferred_lft 6398sec
    inet6 fe80::28c2:db52:8d85:21e/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

This will be different for you, but you’ll have to identify the common network to which both your computer and mobile is connected. For me it’s the Wi-Fi, which corresponds to the wlp3s0 interface, and the IP address is 192.168.0.100 (after inet). So I openthe browser on my phone and go to 192.168.0.100:3000 and I see the HTML page

What a beauty!

Next I write the name, and click “Pair” and I get a notification on my laptop. I click Accept, and start writing in the textarea, and I get the output on my laptop. Neat, isn’t it?

(Hopefully) in the next issue, we will handle mouse events too! Or probably I’ll make it into an app.