Michael Kimsal's weblog - discussion about technology, personal and professional development

Detroit’s been having a tough time of things over the past couple of decades. Industry closing, people leaving, rotting infrastructure, etc. They’ve even been taken over by an emergency manager appointed by the state. The problems are legion, the proposed solutions are all over the place. My humble proposal is short, and to the point.

A state tax holiday for people living in Detroit.

The specifics may be a bit up for debate (5 years? 10 years?) but at the core of Detroit’s problems is a lack of people, and specifically a lack of young people earning money. People don’t move *to* Detroit – they move to the suburbs. Why? Lower crime may be one reason, but typically the issue is jobs and lower taxes. You have to pay a city income tax to live in Detroit, on top of state taxes, and federal taxes. The state has a big interest in getting Detroit in the right direction – instead of being a resource drain. So… the state should give a tax holiday to anyone living in Detroit for, say, 5 or 10 years.

There are people who would move in to the Detroit city limits immediately to save a thousands of dollars in state income tax. Detroit would get income tax from those people to help fund the city improvements that are needed for those areas. The state would lose revenue from those people, but would, over the long haul, be required to spend less to sort out Detroit’s problems and prop them up when necessary – the residents themselves would be doing so.

More people moving to Detroit in the short term would probably mean more commuting – people may drive to Royal Oak or Ferndale for their jobs, but live in Detroit for the tax savings. But over time, more people living in the Detroit city limits would mean more demand for businesses and jobs to locate in those borders as well.

Why should *businesses* get tax abatements and deals to move in to Detroit (or any city) but not residents? Detroit needs more residents than it does businesses. The more residents that it has, the more businesses will follow to serve those residents. An extra 50,000 people living in an area of Detroit should be enough to get some Kroger stores to open up to serve those residents, right?

Yes, this is overly simplistic, but it’s also something that shouldn’t require a lot of planning. People fell over backwards trying to take advantage of the ‘new home buyer credit’ a few years back, which essentially just saved them a few thousand dollars one time, while generally saddling them with huge mortgages. People move to states at least in part because of income tax codes (obviously not always, but for many people it’s a factor). Detroit needs active, productive people to live there. Give them an incentive to do so and I believe they will.

Share and Enjoy:

Convert docx to pdf on OSX

April 8, 2013 Mac / Software

No comments

A client needed a way to batch convert DOCX files to PDFs on a Mac, and I poked around for a few minutes and came up with this:

/Applications/LibreOffice.app/Contents/MacOS/soffice.bin –headless –convert-to pdf:writer_pdf_Export –outdir ~/Documents/ ~/Documents/*docx

This was largely inspired by this post, but LibreOffice is still referred to as ‘soffice.bin’ on OSX (at least, my versions) so this will be easier to copy/paste for me, my client, and others who are searching for this in the future.

Share and Enjoy:

Why do no almost no web frameworks come with any authentication/authorization functionality?

February 21, 2013 Software / Web

8 comments

This is likely a controversial stance to take, and I’ll very likely get bashed as “no idea what I’m talking about” by people much cleverer than me. With that said…

Why do almost no web frameworks provide any default authentication/authorization functionality, with default examples of best practices for common use cases. The standard response I’ve gotten for years was/is “well, everyone’s needs for authentication are different”.

No, they are not. A (very?) large majority of web applications (which is what most web frameworks are used to build), require some form of user login and authorization management, and often self-registration, dealing with lost passwords, etc.

Yet somehow, everyone’s essentially forced in to writing their own user login and management from scratch. This leads to potentially loads of security holes from people writing insecure code.

So many frameworks promote their routing and database layers, configuration management, etc.; those are all things that one could argue people might need to function “differently” – in many cases, the default code is configurable enough to handle many edge cases. In rare cases when the stock code can’t handle things, you can override it with custom code.

But with authentication/authorization, everyone is left to fend for themselves. Every. Single. Time. And they *often* get it wrong (sensitive info in a cookie, unencrypted passwords, etc).

Put another way, when left to fend for themselves, developers need to learn a lot of concepts. Every decision point is a point that can be made wrong (or poorly). Making a poor decision about your CSS colors or URL structure or JavaScript helper library might be painful or annoying, but will likely not have any major repercussions. Making a poor decision about authentication can be devastating. Yet, somehow, this is one of the prime areas in the web framework world where users are not given anything out of the box (in most cases, at least) and are left to ‘educate themselves’ (with quite a lot of bad, wrong or outdated information floating around).

If you’re not going to ship some basic authentication/authorization functionality with the rationale that not everyone’s needs are 100% the same, perhaps you should stop shipping routing, forms management, database libraries and more – after all, someone might want to do it their own way. Not everyone’s queries are the same, don’t you know.

I titled this post “almost”, because I’ve got a hunch there may be a few that I don’t know about. With that said, what web frameworks do you know of which ship with authorization/authentication out of the box? My own experiences indicate:

In the PHP world, it looks like Symfony2 ships with an ACL component, and the recommended ‘default bundle’ distributions ships with Authorization and Authentication components out of the box. Zend Framework ships with an ACL component as well, but in both cases (ZF and Symfony) there is no default way of allowing users to register/login/reset passwords, etc. FWIW, the Symfony approach of distributing recommended bundles of packages which (from what I can tell) could be updated independently if and when need be might be the best middle ground I’ve seen so far. ”Decoupled but packaged”.

The Rails community seems to (nearly) universally rely on Devise, but it’s not shipped by default, and many people end up ‘rolling their own’ (probably very often with bad and possibly even hard to spot flaws).

Grails users often rely on the Spring Security plugin, but again, not a default plugin. To its credit, there is a basic user/role management screen with searching, account disabling, and other maintenance functionality, and the basic system allows for user login and ‘lost password’ pretty much out of the box (self register is a bit more work). But again, not shipped with the base, and people may be tempted to roll their own (although a default ORM means people are far less likely to be susceptible to SQL injection vs building SQL by hand).

ASP.NET ships with a membership system (though it’s been a long time – my memory may be out of date), with web controls for user login, registration, lost password, etc. Whether it’s necessarily the ‘best’ security approach is not really the point here – it’s a standard that is provided, and more than likely has prevented people from (re)writing code in an insecure manner.

What am I missing?

UPDATE: One thought is that no one wants to be even remotely possible for providing out of the box security because they’re afraid they’ll be a target for a lawsuit. I suspect that’s not really a factor, but perhaps it is in some cases?

Share and Enjoy:

Tech partisanship doesn’t help much

January 10, 2013 Development / PHP / Software / Web

No comments

I shared a small rant at a local web meetup the other day, and thought I’d recap it here. It was triggered by sitting in on an interview with a local developer for an idea-stage startup I’m counseling.

The startup in question has had some false starts on an MVP before, once in ASP.NET, and once in PHP (or maybe twice). What’s come out of this was an understanding that the particular tech chosen is less important than finding sharp people who can see the project through to the first phase being functional. I helped write a Craigslist ad to that effect, it was posted, and we’ve sifted through some of the applicants and did a couple interviews.

What’s been interesting to me is that many of the people applying are .net developers. I’ve nothing against .net, but typically, you don’t find too many looking to jump in to the entrepreneurial space; it’s very much a corporate-enterprisey sort of thing. I know .net-startups exist, but they’re a small fraction compared to the number of web startups based on dynamic languages (php, python, ruby, etc).

So we met with someone, and he asked what the site was written in. My friend explained there’s no current site, but an early draft had been done in PHP. The .net-dev grimaced. “PHP is just not capable, it just won’t work for some things, it’s not efficient, there’s things it can’t do, C# is much better technology,” and so on. This was not a 5 minute diatribe, I’ll grant you, but it was long enough, and … ill-informed enough to make me question his judgement skills on other things.

I can expect this sort of tribalism from someone just starting out in development. I don’t expect it from someone with 15 years of experience. I would expect that you’d mature to the point where you understood that almost all tech out there is suitable for a wide-range of applications, and that the language itself is rarely a bottleneck (compared with database servers, network latency, etc.) And yes… I expect this because I’ve seen these attitudes diplayed out over many colleagues, but also definitely in myself.

I’m ashamed of some of the partisan writings I posted back 10 years ago; I was wasting huge amounts of time arguing pointless stuff that had no real impact. Perhaps that’s half the purpose of internet forums in general, but it took me a while to realize the emptiness in those pissing matches, and regret much of what I contributed all those years ago. But it does help give me a sense of perspective, as does my shifting tech skills: I’ve gone from MS/VB/Access to Notes to Perl to PHP to ASP to PHP to Java with a bit of C# thrown in over the years. After you’ve done this long enough, you realize that there’s enough change in tech that few of the skills from 10 years ago really matter, which is another way of saying that little of what you’re learning now will have much utility in 10 years.

I’ve got another rant on reusing software coming up soon; stay tuned (or come back, or subscribe to the RSS) to have a read.

Share and Enjoy:

freelancer? join my community

November 16, 2012 Business / Ideas / Software

No comments

I’ve recently set up a site intended to be a broader mix of content and community at http://freelancepath.com. This is an extension and outgrowth of an earlier email group I ran based on audience members from my web development podcast.

If you’re a freelancer, I’d like to invite you to join my community for web freelancers. Interested in contributing a guest post to our blog? Register there, friend my profile, and send me a message with your idea. I’d love to have you guest blog about your freelancing experiences, tools, service reviews and more!

Share and Enjoy:

Things a web developer might need to know

October 28, 2012 Business / indieconf / Javascript / Software / Testing / Tools / Web

10 comments

I saw a post on reddit the other day, and answered there, but thought I’d clean up my response there for here, fleshing out a couple more ideas, and get your feedback as well.

The original question from was a 16 year old who’s been doing some basic CRUD apps, but is getting bored and wanted to move on to ‘real’ development. There were some good replies, but few went in to the depth of detail that I think beginners are even aware of. Granted, this might scare off some, but for others it might give them some ideas about what’s possible and what’s involved in professional web development. I know I’m going to leave off some topics, so feel free to add on!

Generally, in professional web programming gigs, the “programming” part is just that – a part of the job, and often not even a majority, unless you’re fast and loose with the definition of programming. I’m taking it to mean primarily one language – usually a server side tech like C#, PHP, Ruby, etc.

Version Control

Understanding the basics of version control – when, how and why to use it – is essential for professional software development. Git and subversion are probably the most widely used today – mercurial, darcs, cvs and others are either gaining or losing ground daily, but understanding the basics of git and svn (differing systems certainly) will stand you in good stead in 2012, 2013 and beyond.

Even working by yourself you really should be using it as well, but I frequently talk to solo developers who say “well, I don’t need it, because I just work on my own projects”. A few reactions I have to that sentiment are:

Much like backups, you won’t really understand how much you need it until you need it.
Branching opens up a whole world of possibilities in your approach to development, allowing you to work non-sequentially when necessary, that you’re only thinking with part of your brain without version control.
Most professionals use some form of it. To work with anyone else, you’ll need it, and you may as well start now.

Ticket/issue systems

I don’t have a horse in this race specifically, and personally am not a 100% convert, but the more I work, the more I need things written down in a centralized place which others can use and modify, but that also allows me to hook in to with my code. Being able to commit code and indicate “this is for ticket #723″, and having that tie in to the ticket system so that I can see the code from the ticket system, is very powerful.

Go back to issues 6 months later, and see the code changes in context with the notes on the issue in question – it gives you a different (new?) perspective on how you write commit messages, what’s important to note, what’s not, and so on. Personally, I’m using redmine right now, but have used other tools in the past. Find something that works for you and/or your team and stick with it.

Testing

Unit, integration, load, performance, scalability, acceptance – there are loads of ‘types’ of testing, and you may lump some together, and your process may change over time. I’m less concerned with whether you have load/performance/scalability testing processes – those aren’t always considerations for projects. Unit and/or integration testing are generally useful regardless of the size/scope of the project.

Get comfortable with a testing tool (junit, nunit, phpunit, cucumber, rspec, etc).

Continuous integration

Hand in hand with testing is a way to automate the testing process. Every time you check in code, have a set of tests run and show you the results. Again, once you make this a habit, it can be very powerful.

Jenkins is the current standard in the Java world – there are probably others for other technologies – search for “<my tech language> continuous integration” for specifics.

Security

Along with other types of testing, you should be aware of security testing strategies to employ against your sites. Mess with URLs, try to POST bad data to your scripts, etc. Automate those tests. Find tools to do the same. sqlmap is a tool to automate SQL injection attacks against your site – using that is eye-opening.

Are you using prepared statements over raw SQL strings? Stored procedures? Various levels of access to your database(s)? There are a number of techniques to help avoid or reduce SQL injection attacks.

Learn about Cross-Site Request Forgeries (CSRF), and how to protect against them. Learn about Cross-Site Scripting (XSS) attacks.

SQL injection, CSRF and XSS still make up the vast majority of security holes in websites. Learn how to protect against them and you’ll be a long way towards being secured (but never take it easy!)

Performance

There’s a whole world of topics to cover under performance – code caching (do you write optimized and optimizable code?), data caching, page caching, HTTP caching headers (etags, etc), asset caching, compression, minification, CSS sprites, mobile-optimized sites.

As I said before, many of these may not be useful to all developers all the time – they may never rise beyond the level of ‘interesting’ at your current project/gig. Be aware that the tools, techniques and trends may change quickly as new tech and usage patterns emerge, so even if you ‘know’ this stuff, revisit it every so often if you’re not immersed in it day to day.

JavaScript/front-end

How good are you with JavaScript? Would you be able to write a full app in a browser using JavaScript only, making service calls to a back-end via SOAP or REST? There’s a whole world with toolkits and libraries like jQuery, Dojo, AngularJS, JavaScript MVC, templating systems and more. Are you able to selenium-test your front-end app? How about running browser-based tests via qunit or a similar testing tool?

Mobile

The rise of mobile – smartphones, tablets, etc – has opened up a new set of opportunities and challenges to be aware of. Data caps, optimized graphics, new UI controls for touch interfaces, and more. Understanding ‘best practices’ for mobile, and keeping up with them, will keep you busy for a while.

Other technologies

How good are you with search tools? Business dashboards? Data gathering and analytics creation, interpretation and action? There are a number of things that businesses need which don’t particularly relate to any one specific tech, but they’ll all need (quick way to search for data, generate reports, etc.) Find some common business problems in your current situation and look for some of the top packages out there that solve those problems that you can integrate (SOLR, Lucene and ElasticSearch on the search side, for example; Jasper Reports or Pentaho on business reporting options, etc.)

What language again?

Notice that I really didn’t focus on any language or particular tech. All of the above are skills that professional web developers need to have – or, if not possess 100%, be *aware* of. I’m certainly no master of web tech, but I keep up with it enough to know who the real masters are in various areas.

What surprises me some is students coming out of school, and sometimes with more than a couple years under their belt, who’ve never heard of some or many of these ideas. Perhaps I’m just meeting more than my fair share of true ‘code monkeys’ who copy/paste PHP/jQuery from 9-5, but I’d like to think, but that initial reddit post got me thinking a bit about this (that and some recent conversations with beginners and seasoned experts at a few regional meetups lately).

Won’t this all change?

Yes and no. The idea of continuous integration was certainly not popular when I started in software development …. 18 years ago. No doubt it was being done, but not by people I knew, nor in any popular literature I could find. Some of these ideas take hold, and some don’t; Test-Driven Development, ‘Agile’, etc may come to be seen as fads in a few years – I can’t say for certain. But… the fundamentals of communication and being aware of multiple aspects of a project (accuracy, speed, security) won’t go away. These are issues that *will* be addressed on a project eventually, either during the initial work when it’s under your control, or at 2am on a Saturday morning because everything’s broken or you’ve been hacked

Gentle plug: if it’s before November 17, try to make it to indieconf, a conference for independent web professionals (and maybe just those that act like it!)

Side note: one of the upsides of freelance work is you often get to control the tools/processes for the work, and can pick/choose the tools you want. One of the downsides is that sometimes you end up working with a team who “doesn’t believe” in any of this stuff, and you end up wasting a lot of time fighting problems that continually get reintroduced because of lack of testing. I’m no saint on all this – I’ve done my share of skimping, and I speak from experience when I proclaim the value of using these sorts of tools.

Share and Enjoy:

Code testing

October 16, 2012 Development / Software / Testing / Web

No comments

The TriJUG last night had a speaker cancellation, and someone from the group presented an ad-hoc talk on testing – outlining various types, pros/cons, some metrics and code samples he uses, etc. Overall it was useful for a topic, and I hope he polishes it up and presents it next year as a fuller talk more fleshed out.

I’d interrupted a couple times with my own questions and experiences, but there was one I wanted to share which I didn’t last night. He was talking about the justification for testing (unit and integration primarily), and couched it in terms that developers would be familiar with. ”Without tests in place, you can’t be confident that your changes won’t break anything!”. It’s true, and for some audiences (last night’s) it makes sense.

Let me preface this by saying that I do not do enough testing myself in my projects – this is a ‘do as I say, not as I do’ post, somewhat, but hear me out. :)

There’s a more fundamental reason to ‘do testing’, as it’s often referred to by non-techies (and even some techies I know). In fact, you can’t *not* test, which is the fundamental point I was going to make last night. Testing *happens*. *When*, *where* and *how* you test is totally up to you. You can have testing procedures before changes are pushed out, have a test suit to give a sense of confidence to the team, and so on. Or… you can just push code to production servers. It’ll get tested with real world data by real users, with often real (bad) consequences.

Again – *testing always happens*. Testing on your own terms, with defined procedures, steps, processes, etc. is better than testing ‘fixes’ at 2am on a Thursday morning.

“But there’s no time to test!” is what I often get as push-back from people I present to (and, yes, I use it myself sometimes on myself!). Let’s just say, for argument’s sake, ‘testing’ (writing tests, defining them, checking them, etc) will take 100 hours on top of ‘development’ (as if they’re somehow distinct entities!) If you skip those 100 hours, you’ve only shifted them around. You’ll spend *at least* 100 hours cleaning up stuff after the code has gone live, and in the process you may damage the reputation of your department, company or yourself in the process. No amount of time (100 hours? 1000 hours?) can replace the damage that’s done by lost orders, lost money, privacy exploits, security breaches or worse.

So to reiterate – the testing *will* happen. Under what circumstances do you feel most comfortable testing? In non-public controlled situations? Or out in public where your customers and competitors can see and exprerience your mistakes?

Check out the conference for web freelancers while you’re here…

Share and Enjoy:

web freelancer conference update

October 10, 2012 Conferences / indieconf

No comments

indieconf, the conference for web freelancers, is coming up soon. We’ve got our schedule nearly full (couple of last minute changes coming up soon) but it’s shaping up to be at least as good as last year’s event.

The schedule includes a wide range of topics, including “Understanding the Money Side of Your Business“, “Monetizing Your Websites“, “The Art and Science of Creating Work That Matters“, “Practical Copyright Enforcement“.

Our speakers include some indieceonf veterans like Patrick O’Keefe, Bermon Painter, David Rogers and Doug Foster, along with new faces like James Clear, Katie Benedetto and Leslie Byrd.

This will be our 3rd annual indieconf, and it should be the best one yet, but it won’t be without you. Sounds corny, right? But it’s true. The speakers are great, but what ends up making each event special is the attendees and the connections people make there. If you’re a freelancer – designer, developer, writer, project manager, you need to be at indieconf. Our sessions will increase your business, and our social aspect will benefit your network.

P.S. I often get asked ”do I have to be a tech/geek to attend?”. You can click that link to read the longer answer, but the short answer is ‘no‘.

Share and Enjoy:

Open Source LinguaFolio software project available

September 4, 2012 Business / Development / Grails / Groovy / MySQL / opensource / Software

No comments

A couple of years ago I was asked to build a piece of software implementing the concepts of the LinguaFolio “formative assessment tool”. That’s education-speak for something that helps students and teachers understand how a student is doing. I was asked to build this by the state of North Carolina, and for the past two years they’ve been rolling this out to more students and teachers around the state.

The LinguaFolio tool is free for all students and teachers in NC, and is essentially free for others to use because it’s an open source software project. The main code is available to download and run anywhere. It’s a Java/Grails/MySQL web app which, while not as simple to set up and host as a PHP app, is still pretty basic. We’ve still got a lot of functionality to implement, and I’m slowly adding some new features over the next few months, but I thought I’d mention this here so people would know it’s available. As far as I know there’s only one other ‘online’ version of LinguaFolio, but it’s not open source or free or even available to test out. As more people become interested in using LinguaFolio, there may be more options, but right now, our elinguafolio project is picking up steam.

Now you may be thinking “hey, that’s great it’s open source, but I don’t know how to run any software like that!”. And you may be right. Just because something is open source doesn’t necessarily make it 100% useful to anyone out of the box – it’s more a case of giving you freedom to find the right vendor to work with (or change vendors if you’re not happy). If you’re interested in using LinguaFolio in your district or state, tell your vendor to grab the software and run it. They can reach out to me with questions (there’s an issue tracker on that site above) and, within reason, I can assist them in setting it up.

Or… you can simply hire me to do it for you. I can set it up and host it on servers I manage (and can give you access to as well) for a standard monthly price. You’ll have a version that is kept up to date, and you’ll be helping to fund the open source LinguaFolio which will continue to help more teachers and students over the coming years. If you’re interested, give me a call at 919 827 4724 to discuss how I can help.

Share and Enjoy:

a dumb programming mistake

August 29, 2012 Database / Grails / Groovy / Java / PHP / Software / Testing / Web

3 comments

I had a couple of dumb development mistakes come and bite me in the butt this week, and I thought I’d share them here – well, at least one for now.

Without going in to the type of data too much here – the specifics aren’t important – the system I’ve got needed to show a list of data to a user. Initially we just showed all the data to the one user. Later, we added other user types. However, my controller code (MVC-style) simply defined different views to use based on user type, but never changed the query. So, whether you were an admin, or an area user, or a distinct individual which should only have seen a few items from the list, your view was passed all of the data. The views simply filtered out which ones to display during a loop.

Stupid? Yes. Noticeable? Not at all. I looked at this code yesterday and remember thinking at the time “yeah, I’ll go back and make this better later” – I was planning on overhauling the user/role system, and I’d fix it all later. That never happened, and over the past couple years, this has become slower and slower.

Fast forwards to yesterday, and we had 80 processes each being handed *290,000* objects ina view, in most cases to filter and display, say, 60. Even getting new updated software on the server today was a pain, because we were continually hitting loads of 40-70 (unix loads, 15 min avg of 60 was where we were at most of today).

This came to a head the past couple days because our usage patterns changed – user load spiked both because of timing in the month and the user base has grown. It was a perfect storm, and I was caught in the middle. The upside is that I found this major issue and it’s resolved. The downside is that it took almost 2 days (mon and tue) because I was finding other things I thought were the culprits; they were issues, but had no major impact given the 800lb gorilla in the room.

Lessons learned? I’m not immune to even basic stupid oversights. The moment I saw this code, I knew the issue, but I’ve not looked at it in… 3 years? There was no need to, so I never went back to audit for stupid code. Perhaps I need to budget more time for stupidity audits?

Any major faux pas you care to share or admit to?

Share and Enjoy:

Michael Kimsal’s weblog

Solution to Detroit’s current problems

Convert docx to pdf on OSX

Why do no almost no web frameworks come with any authentication/authorization functionality?

Tech partisanship doesn’t help much

freelancer? join my community

Things a web developer might need to know

Code testing

web freelancer conference update

Open Source LinguaFolio software project available

a dumb programming mistake

Recent entries

Recent comments

Topics

Meta