I have always wanted to revisit the idea as I felt like it could be super cool to have an emulated terminal in the browser, with my blogposts simply part of the filesystem. This week, Coder Story asked to interview me, and one of the questions asks me to talk about how I learned programming and how I develop my projects. I pride myself on the fact that I revisit old projects to improve them, and realised I never did for this one. I felt that it was the perfect time to go back, and that the process could be very valuable for CoderStory readers to see. I decided to spend a few days taking the time to recreate the project with my current skills and tweet about it the whole time.
I started with tweets announcing what I was going to be doing, and describing the old concept and the plan:
The project I'm going to be revisiting was actually my first attempt at HTML, written just before I started University 🎓 in 2014.— Jetroid (@JetroidMakes) February 10, 2019
I had just gotten into Linux 🐧, and wanted to play with HTML, so created an amateurish terminal-themed 'blog': https://t.co/91e6ATTNwT pic.twitter.com/Qts3jSsd2L
I think it’s generally a good idea to draw some sketches of your idea. It gives you something to compare against. Some people go through painstaking detail to draw their interface in design software and ensure their site matches it pixel-perfect, but that isn’t my style. When I’m building something complicated, I sketch out the architecture of what I’m building in a block diagram format. I also like to sketch out the database tables to reduce problems later.
In this case, I have a good idea of what I will be building, and I have the perfect reference image in the form of my terminal which I can open at any time. This project was so small that it was hard to go wrong with the plan, but I still wanted to sketch out some ideas so that my mind would think about how I was going to implement things. The problem here was that the project could go on forever (there is almost unlimited possibilities for command line programs to implement!) and I could keep adding ‘one last feature’. The plan for me served more to limit the scope than to clarify the idea.
I'd like the new version to be more interactive - there should be a handful of 'programs' ⚙️ (functions) the user can 'execute' and a 'filesystem' 💾 (just json) that they can navigate.— Jetroid (@JetroidMakes) February 10, 2019
This is a small project so doesn't need much preparation, but I sketched my ideas on paper: pic.twitter.com/Wn4xnEblvT
It’s quite neat that the sketch of how my filesystem would work turned out to be very similar to the actual implementation (though I did end up switching to an object-oriented style of declaration to reduce repetition and avoid typos).
I have some of my 'filesystem' 💽 in my HTML file and some in my JS file. The stuff in the HTML file lets me use Jekyll to 'pull in' the metadata about my blog posts when the page generates. I'll be adding more here later, this is just a proof of concept. pic.twitter.com/oWxYWhMpbb— Jetroid (@JetroidMakes) February 10, 2019
My first step is usually to create something that I can build from. In the case of this project, it was a basic, ugly command prompt that I didn’t put any effort into CSS styling. I decided that the layout of my real command prompt would be the layout I would use for the prompt in the website. My command prompt layout is custom and has a lot of information, so gives some visual interest. I then built methods like
print() that a command could run to write to the output, and an
enablePrompt() function that indicates that the command is done, that the prompt should be shown, and that the user can start typing again.
(At some point I went back and modified
print() to escape HTML entities (eg convert
<) in the user input so that typing something like
<script>alert(0);</script> doesn’t get parsed as HTML and then executed, instead printing
<script>alert(1)</script> which is visually identical but is parsed differently. I sometimes needed to add HTML of my own in the print, so created a
printUnsafe() which doesn’t do it for you, and expects you to escape any user input that you use. In a real site, this would be referred to as “protecting against Cross Site Scripting (XSS)”, and is very important to protecting your users. Often you want to escape HTML entities AS SOON AS THE DATA ENTERS YOUR SYSTEM. Here, XSS isn’t important at all as a user cannot change the state of the page for future users, but I did it anyway so that my more nerdy friends wouldn’t make fun of me.)
First step is to set things up so that we can print to the console when the user hits enter. It's a simple first step but a necessary one. I wanted to use the DOM's cloneNode(true) but it would have created duplicate id's. 🙅♀️— Jetroid (@JetroidMakes) February 10, 2019
Next step: Parsing input and executing it. ✌ pic.twitter.com/xH4Zrnfdx9
To expand on the
cloneNode comment from the previous tweet:
cloneNode(true) makes a deep copy of the DOM elements, which would be useful as I wouldn’t have to ‘recreate’ the prompt when I add it to the history after the user presses the enter key (return key). However, cloneNode makes a truly accurate copy, meaning the ‘id’ property will be duplicated, the event handlers will be duplicated, etc. We don’t want any of that, which means we can’t really use cloneNode, and we have to do it from scratch ourself.
After I had programmed the file system (mentioned earlier), I made a few helper functions to convert objects to paths, and paths to objects. If a user specified any path with any valid format like:
…then I can return the filesystem JSON object that represents it. Useful!
Wrote a function to do the opposite of the above.— Jetroid (@JetroidMakes) February 10, 2019
Looks simple, but I had a bug in my while loop for a while and I kept infinitely looping and crashing my browser. 🧐 🤔 pic.twitter.com/6TZ5DOZ4G8
This meant that when a command referred to a file or a directory, I could easily access the file. Likewise, it meant that I could easily add the working directory to the command prompt.
ls, two commands that involve interacting with files, were also easy to implement with minimal code.
Those functions worked to implement `cd` and (the most basic version of) `ls`! 🎉— Jetroid (@JetroidMakes) February 10, 2019
Just for fun, let's do wget next! 😄 pic.twitter.com/eXPjSnRq5W
I wanted some way for users to create files (but wasn’t prepared to implement a more complicated program like
vim for this sprint), so I decided to create
wget, which would actually just be a AJAX call to a specified website.
I ended up having problems with CORS. CORS is a security policy which prevents sites from making requests to servers on a different origin. The browser refuses to connect to a service which has CORS headers in the response. I don’t think CORS is a very sensible mechanism, as it does not prevent the request from being made, it merely relies on the browser not making the request.
curl do not resepct CORS (hence CORS is not a valid mechanism to protect from server-to-server requests), so my implementation of
wget should not obey CORS. Setting up a server between the resource we want to request and the browser would allow us to proxy the request back to the browser without the CORS response headers, hence circumventing the CORS protections - hence why I think CORS is a pointless protection! CORS Anywhere already lets you do this, so I deployed it on Heroku, changed my code to proxy through the dyno, and just like that, my
wget command worked.
Hurray! `wget` is now working too! 🙌 (Or at least, the most basic functionality is working - I still need to update the text and check for bugs.)— Jetroid (@JetroidMakes) February 10, 2019
I'm able to download files from both https://t.co/SU3VrtBYfn and https://t.co/KHW5t0jzzF (the two links I tested). 😊 pic.twitter.com/uISasPCsSh
In the morning, I updated the CSS to move away from the horrible white background and black text towards a colour scheme that matched my real terminal. I remembered how some servers have ascii art in the header after you log in, along with warnings about monitoring users. I also remembered how in the media, a 1337 h4ckz0r might have their online alias in ascii on their personal terminal or server. I decided to write ‘Jetroid’ in the style of my logo in ascii art, and I think it turned out pretty well!
Added some fluff to make it look more like you just logged into a neckbeard-admin's server. 😅— Jetroid (@JetroidMakes) February 11, 2019
Obviously today I've broken protocol by focusing on front end visual rather than functionality, but the white background was giving me eye strain. 😳 pic.twitter.com/lFQN3kshB9
I then implemented ‘animated’ commands - commands that initially give out a little output, but then add more quickly as data is loaded or processes complete. I did
shutdown. I’m a big fan of
shutdown because I added jokes playing on the
Stopping <daemon x> and
Shutting down <process y> - it may initially look normal because the first few are ones you’d expect to see, but then it devolves into being entirely jokes. My favourite is
Shutting down the-us-government, and both my friend and I simultaneously came up with it at the same time. I think the end result is very authentic whilst also being funny.
Back again after eating 🍜 and I've implemented a `shutdown` command.— Jetroid (@JetroidMakes) February 11, 2019
I wasn't able to find anywhere to copy some of the shutdown text so I had to write my own from memory. To bulk it up I added plenty of jokes. 😅 pic.twitter.com/foIsD3UgHB
In the morning, I implemented
cat for my blogposts. To save memory and decrease page load times, the content of a blog post isn’t stored in full in the content of the /terminal/ page. Instead, I request the file from the server when needed using an AJAX call.
But first, some quick code to make sure I can `cat` the HTML of one of my blog posts. pic.twitter.com/oH6Xj7tLb5— Jetroid (@JetroidMakes) February 12, 2019
For the benefit of CoderStory’s readers, I discussed the UNIX permission system. I was trying to show that you can combine knowledge you have with content to try to gain new information. I also wanted to show how I had questions about the UNIX permission system, and that by writing them out, I could find answers. This is a big part of learning to code, so I thought it important to mention, even if the example is not the best.
The nice thing about writing out what you need to know like this is that these are basically questions I can throw into google...— Jetroid (@JetroidMakes) February 12, 2019
16:57:44 jetroid@netricsa:0~/projects/jetroid.github.io$ groups
power wheel input storage users pic.twitter.com/guDHqCPrDT
I used this information to implement read/write permissions for commands like
rm, and others.
I also implemented read/write permissions for use with cat, wget, and others (when I implement them!).— Jetroid (@JetroidMakes) February 13, 2019
`jetroid` can't read /etc/shadow nor write to / 🙅♂️, but root can do both ✅.
You'll see the precedence described at https://t.co/DhFKgxYdHn quite clearly in lines 212-217. 😁 pic.twitter.com/19HJ3jyNie
I implemented functionality for superusers and
sudo simply sets a variable to indicate that we are in
sudo, whilst superusers have an
isSuperUser property. I modified the commands to first check for
user.isSuperUser to bypass checking permissions. I then implemented
su, which pushes the current user to a stack (so that we can return to it when we run
exit), and then changes to the desired user.
sudo is kind of neat because it just recursively calls
determineCommand and has no problems.
I then went ahead and implemented a whole bunch of other fairly simple (< 5 lines each) commands that either interact with the filesystem permissions and user system (like
whoami) and ones that were simple to implement (
Here's a little demo of sudo and su, as well as a few of the other commands.— Jetroid (@JetroidMakes) February 12, 2019
Note that as `jetroid` chown was denied, but it was allowed as both a `sudo` command (using !!), and with `root`.
It looks so much like my terminal right now that I keep trying to use my shortcuts. 🤐 pic.twitter.com/dYeAQUCReB
When I encountered a bug in my code, I wrote about it and my debugging process:
I was confused, so added logs to the `rm` function (blue circle), then reran my command. I saw that `rm` was being called as if I had first specified the root `/` (green) by writing blank (orange).— Jetroid (@JetroidMakes) February 14, 2019
I suspected multiple-spaces in the middle of a command, so I tested it (red). pic.twitter.com/q6comUAB6F
We can solve that first point pretty easily by sanitising our input a little better, and replacing repeated spaces with a single space, then trimming off all whitespace to the left or right of what the user wrote.— Jetroid (@JetroidMakes) February 14, 2019
It's a pretty simple fix, all things considered. 👍 pic.twitter.com/8OOpzkuD1y
But there was also a second 'bug' - did you notice it?— Jetroid (@JetroidMakes) February 14, 2019
The natural whitespace collapsing is affecting the 'past terminal entry' in an undesired way! Multiple spaces got represented as a single space! 🤦♂️
To fix this, we can wrap it in a <pre> tag or use CSS white-space:pre; pic.twitter.com/758f2xMOdj
I was approaching the end of the project, so I decided (for fun) to implement a very simple command that converts the styling to be more like the original iteration.
The biggest feature that I had been dreading was making the typing look correct, aesthetically. Up until this point, the page used a boring text box and a button for input. I wanted to change that so that it was indistinguishable for typing on a normal terminal. When I had worked on the old version, I tried everything I could to make the
<input /> or
<textarea /> tags flow to cover the rest of the page, but I wasn’t able to and it was still quite noticeable what I was doing.
I wanted to avoid that, so figured that I would try to do some fancy
This was a BAD IDEA.
As it turns out, without a library to help you, the
e.which library is fairly messy to determine which key you are using. Different keys on the keyboard are bound to each of the
onkeydown etc functions without any rhyme or reason that I could see. And you manually have to keep track of which modifying keys (like Ctrl, Alt, etc) are being pressed.
I actually got something that ALMOST worked, but if I typed too quickly, the characters would appear in the WRONG ORDER. UNACCEPTABLE.
In the end, I decided to resort to a much more simple method:
Then all it takes is a little function to take the data from that input tag and display it where I want, which I call onkeyup and oninput. A final onclick to the whole body to autofocus on the 1px by 1px text input completes the illusion. 🧙♂️— Jetroid (@JetroidMakes) February 14, 2019
And it works really well! 😄 pic.twitter.com/0W2AjaOaJF
After this, I considered the project “done”. Obviously, I could extend it almost infinitely and implement things like bash file parsing or whatever, but there has to be limits or you’ll just waste your whole life.
I added a few finishing touches and signed off…
Some final statistics:— Jetroid (@JetroidMakes) February 14, 2019
🕖 25 hours (how many composing tweets? 😝)
⌨ 1223 lines of code
🐦 60 tweets (including this one)
💾 15 git commits
🤖 18 commands
🤪 Too many emojis...
Crazy ride! Thanks for following along!