Typical Programmer

A glittering performance of rare perception


Popular articles
All articles

About me
Hire me

Things you need to know to do web development

01 Mar 2015

Here’s a list of things I know, or at least know about, as a web developer. I’m sure I’ve left a lot of things out. Web development is a large and complex collection of technologies, tools, languages, protocols, and services. I started programming for the web back in 1995, so I’ve been able to adapt to changes and learn new tools as they were released. If I had to learn web development from scratch today I’m sure it would take me a long time to master even a few of these things.

If you are doing a new project from scratch you’ll get to pick and choose your tools and “stack.” If you are coming into a team that has already made these decisions you’ll have to adapt. If you work with legacy code you’ll have to get familiar with more languages, frameworks, libraries, technologies, and programming styles.

Web applications usually have two big pieces: the front-end code that runs in the browser, and the back-end code that runs on the server.

The front end (browser side)

You’ll need to know HTML, in detail. HTML5 is the latest but you’ll probably have to know how it’s different from HTML4. You also need to know how styles work: CSS, the box model, inline styles, inheritance, specificity, resets, the differences across browsers and browser versions. You will probably need to know about grid systems, and responsive layouts. Most likely you’ll need to learn one or more of the popular front-end frameworks: Bootstrap, Foundation, Toast, Yaml, etc. You may have to know about CSS preprocessors, like Less and Sass.

Almost all modern web applications use Javascript, a lot, so you need to know Javascript. You will probably have to learn jQuery because it’s everywhere. There are other Javascript frameworks and libraries: React, Angular, Ember, Backbone, Prototype, YUI, GWT. You will have to be careful that the Javascript tools play well together, and they work with your HTML/CSS framework. All of the popular frameworks and libraries offer add-on modules and extensions for things like input validation, modal dialogs, drop-down menus, animations, and so on. You will have to learn some of those, too. Javascript manipulates the contents of the web page through a standard abstraction called the Document Object Model (DOM), so you need to know how that works. Browser events such as clicks and keyboard presses can invoke Javascript event handlers. To interact with the back-end server from Javascript you will need to understand AJAX. Once you start using Javascript you will want to learn how to use the browser developer tools, DOM inspector, and debugger.

If you’re lucky you have some design skills, or you are working with a designer who understands the web. Otherwise you’ll be responsible for layout, typography, colors, image formats and optimization, for a start. Customers often don’t understand that a web page is not a Photoshop proof, that they can’t put a 40MB TIFF file on their home page, or why their favorite font is not available to everyone else.

You probably have to know something about the different kinds of devices people use to browse the web and how that affects design and front-end implementation. Desktop and laptop computers, tablets, and phones have obvious and subtle differences. There’s no hovering on most touchscreen devices, for example. High-resolution Retina or 4K displays may give you grief. Mobile users often have slow connections and data rate caps and limits, so you need to think about that and test on all kinds of devices, with wi-fi and cellular connections.

Somewhere along the way you need to understand the HTTP protocol, at least what request/response and statelessness means. You need to understand cookies, browser caching, expiration, content types and MIME. You have to know about mixing content retrieved with HTTP and HTTPS. You will probably need to understand synchronous vs. asynchronous requests.

The back end (server side)

The server stores and serves the HTML and CSS and Javascript files to the browser. A web site that only has HTML, CSS, and Javascript is called a static site. Non-trivial web applications have code running on the server to generate the front-end code dynamically based on user inputs, session state, database queries, and external services. The web server may run one of the general-purpose web servers such as Apache, Nginx, IIS, Tomcat, or it may integrate the server with the application, like NodeJS. There may be multiple web servers handling different types of content: proxy servers, CDNs (content delivery networks).

The web server runs code that may be written in any number of languages, and many web applications use more than one language. PHP is widely used, so is Ruby with the Rails framework, Java, the various .NET languages (C#, VB.net, F#), Javascript with NodeJS. You may run into code written in Perl, C, C++, Go, Erlang, Python, or something else. All of the big server-side programming languages have extensive libraries and components you will encounter or want to use.

Almost all web applications use a database or some kind of datastore accessible by the web server. MySQL, MS SQL Server, PostgreSQL, SQLite, Oracle, DB/2 are all relational database management systems (RDBMSs) that use the relational model and some dialect of the SQL language. You may use a non-relational (so-called NoSQL) database, and there are many of those: MongoDB, CouchDB, Cassandra, Redis, Memcached. Many web applications use more than one database or data storage tool. The database management stuff may be partially hidden behind an ORM provided by your application framework, but it’s likely you will eventually have to understand the details of how your data is stored and retrieved, how data integrity is assured, and how the data is backed up and secured.

External services give you access to all kinds of things: sending and receiving email, credit card and check payment processing, analytics, maps and location services — the list is endless. If your web application uses external services you will have to learn about APIs, REST, authentication, JSON, XML, RPC, SOAP. You may have to use or write a scraper or crawler that extracts data from sites that don’t offer an API. You’ll probably have to work with curl/libcurl at some point when getting your server to talk to an external service.

Besides the languages there are programming techniques, styles, and workflows you will have to understand: object-oriented programming (OOP), maybe functional programming, certainly imperative programming, test-driven development (TDD), Agile, SCRUM, etc. You may need to use a debugger for the server-side code.

The web server has to be hosted somewhere. It may be a server in the office, or co-located at a hosting facility. More likely it will be a shared, dedicated, or managed cloud instance at Amazon, Rackspace, Google, Microsoft, or any number of cloud hosting providers. Hosting decisions can limit your choice of languages and tools, how much control you have over the server configuration, and things like email handling, DNS, bandwidth limits, and so on.

Security and legal/regulatory issues

As soon as your application goes live on the public internet it will be attacked by bots and hackers. You have to understand common vulnerabilities like SQL injection, cross-site scripting (XSS), cross-site request forgery (CSRF), cross-domain request issues, CORS, session hijacking, spambots. If your employer or customer doesn’t have a system administrator you will have to know about firewalls, keypair authentication, and securing your servers. You have to keep up with security threats and patches to the server operating system and every part of your stack.

If you accept credit cards or electronic payments on your site you will be subject to PCI compliance audits. Other types of personal data you collect and store may be subject to other compliance and audit rules (HIPAA, COPPA). Every country your site does business in will have its own legal and regulatory requirements.

Miscellaneous

You will need to master at least one text editor. You will have to know how to get your code uploaded to a server with FTP, SCP, or some other deployment tool. You should be using source code version control: git, subversion, Mercurial. Eventually you will have to learn your way around UNIX, the file system, the shell, tools like find, grep, vim, emacs, nano, ssh, cron.

You will have to understand domain name registration, DNS, probably SSL certificates. You will probably want to monitor your server for crashes and outages and spikes in activity than can signal a denial of service attack or something else wrong.

If your site is available in more than one country, or your customers may not all be English readers, you have to know about internationalization (i8n) and localization and deal with translations and multi-lingual content. You will also have to understand timezones and date/time conversions. Depending on where your users are you may have to deal with country-specific issues, like China blocking most Google contents (goodbye Google-hosted fonts and Javascript libraries).

All of the languages, libraries, frameworks, and tools you choose will be updated and patched frequently, so you have to stay on top of that while making sure updates don’t break your application. Even moderately-complex web applications can have tens or hundreds of dependencies at different layers in the stack. Dependency management and managing versions and updates can turn into a risky and complicated problem all on its own, and of course there are tools to learn for that.

A large number of web sites are based on content-management systems: WordPress, Drupal, etc. You may have to know the advantages and shortcomings of those platforms, along with a lot of the stuff listed above.

If your application get significant traffic and users you will need to think about bandwidth (especially costs), performance, scaling. Decisions that made sense during development or with little traffic can come back to plague you when your site gets loaded.