Go, Do, Be.

Categories

09/03/2008  — 

A day on Chrome

I admit it. Upon hearing Goog's announcement about Chrome's public beta, I threw up in my mouth (just a little bit). My learned-reaction comes from too many years wearing the QA hat at consumer-facing websites. The last thing the world needs is another browser floating around... another platform to support... another frigging pile of test-cases to down-prioritize and never get around to running.

I got over it. I spent a day surfing with Chrome, Goog's new browser. Here are a few first impressions.

Starts with a 475k downloadable setup program.

Once it's in, you can import Firefox or IE settings (including cookies) then it starts GoogleUpdate in the background (not surprised).

User agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.27 Safari/525.13

Wow! It's nearly Safari for windows. It's using webkit, already in Safari and sounds like it's going to be used in android. Kinda makes sense to steer users toward a platform that's capable of making the leap to mobile devices.

As software goes, Chrome is handsome. In fact, Chrome is a pretty clever name since there is practically no chrome to the browser. Screen real estate isn't wasted on the borders, status bars, or any of the chrome-y bits that are likely ruining your browsing experience.

Under the hood, Chrome boasts separate process space for different operations. It's trying to get around the single-threaded way most browsers work (without clever webdev hackery). There's also a revved-up JavaScript interpreter... or virtual machine... or something. Need to learn a little more about V8.

The task manager is a nifty idea: show all the processes that are running: tabs, plugins. Kill a process if it's out of control. Be sure to hit the 'Stats for nerds' link on the Task Manager. Is that actually showing memory usage for IE (which also happens to be running on my system at the moment). Yes. Kinda neat. It'll show FireFox too (but not seamonkey!). Helps you 'place the blame where blame belongs' if/when things go wrong.

Chrome's memory stats.

Are you on Chrome right now? Here are a few interesting views under the hood:

My day with Chrome has been a surprisingly upsell-free experience. I see the Google Gears integration, but nothing compelled me to use it. I expected to be directed toward Google Docs... it never happened.

Goog probably gets all of your usage data. Chrome logs everything in your browsing history unless you're in incognito mode. The Google docs describe a scenario where a user might want incognito-mode to order a surprise birthday gift, but I think they meant to say 'download porn'.

"To browse the web without keeping a record on your computer..." I love that. You'll clearly be keeping records on other computers.

Lots more info in the comic book. Allegedly, Goog printed up a bunch of these things and mailed them out. Anyone have one?

Overall I'm impressed. It's snappy, handsome, seems to handle all the torturous pages I drag my browsers across. No heavy upsells. Seems to have a lot of open-sourced projects under the hood. I'll check back in a few weeks.

04/09/2008  — 

bashWebTest Lives!

Well, I did it. I tossed a little pile of code onto google's open source site.

A few years ago I wrote (in my spare time) a little test harness around some simple command-line utilities. I wanted something to help me answer some simple questions about what was going on some large clusters of servers. Rather than clicking through a bunch of nicely formatted pages, I wanted something to make a bunch of http requests and give me a 'yes' or 'no' about the response. The trick (for me) was to try to run it on some server in the cluster which was running a really lean installation of Linux. No frills.

I could have probably compiled a jar & dropped it onto the server... but I couldn't edit & recompile a class on the server. I could probably have run a perl script, (geeze why didn't I just write it in perl?) but I think the WWW-Mechanize module wasn't installed. Who knows... anyway, I ended up stumbling upon curl and decided to write a wrapper around it using simple bash scripts.

Guess what? It worked. It was handy. Guess what? I used it at a few jobs since I wrote it. Guess what? It's still (somewhat) handy. So today (or yesterday) I give something back to the internets and interwebs that haz given me so much. I offer:

Tests are pretty simple. I'll toss a few test examples on this blizzog and onto the wiki on code.goog over the next few weeks. If you have any interest at all in using something klunky, and somewhat functional, please contact me and I'll help you get started.

Code.goog doesn't have a way to select it, but I planned on distributing the source under the 'MSL' license.

Anyway, enjoy!

02/05/2008  — 

QA Interviewing: The Phone Screen

I have a bunch of notes from when I was putting together the hiring process at Kodak Gallery. I dusted these off recently to help a friend screen candidates for a QA position he's looking to fill.

I think back to our problem at kodak: we needed to hire super-geniuses to work on a completely custom-built system. This was before hibernate, before spring/pico, I think it was before the JSP 2.x spec even. We needed to hire people who could 'hit the ground running' and contribute without a hige ramp-up. We needed to perform a thorough search without booking the whole team for all-day interview sessions.

Part of our solution came with a formal phone screen. Once we started applying a standard approach to all candidates we were able to do (something approaching) an apples to apples comparison between the top candidates. We were also able to weed out some bozos who had 10+ years of experience but never encountered an application's error log.

General QA Questions
  • What's the difference between Blackbox and Whitebox testing? Can you give me examples of how you have done both?
  • Can you describe regression, performance, stress testing? Describe when you have conducted each type of testing?
  • When you report a defect, what information to you include?
  • Why choose to include this info? What's the benefit?
  • A new build is available to QA, what's the first thing you test?
  • Have you used automated build & deploy systems?
  • What is an equivalence class?

Through these questions you can get an idea about how seasoned a candidate is: have they encountered a metrics program? Have they worked on a performance testing project? Do they understand the challenges with managing software deployments in the testing cycle. The 'what's in a good bug' question can open up discussion in a number of areas: is the candidate aware of the connection between a feature, the server source, the browser's role... You can learn a lot from this one.

Teamwork Questions:
  • Can you give me examples of your experiences working on a team?
  • Can you give me examples of your experiences working individually?
Sure, this is another section I'll usually skip unless there's lots of time.

Technical Questions:
UNIX
  • How do you find the timestamp of a file in Unix?
  • How do I read a log file as messages are being printed to it?
  • How do I determine the location of a file called test123.txt in a Unix file system?
  • What's the difference between CLASSPATH and PATH?
  • How could I transfer a file from my windows PC to a unix machine?
WEB
  • What is html or xhtml validation? What is the advantage of having pages that validate?
  • Name 4 http methods.
  • What's the difference between GET and POST?
  • If you don't know how to do one of the above tasks, how would you find out?
  • Have you ever used a proxy? Why would this be useful as a testing tool?

The goal here is twofold: first, to see whether the candidate can handle themselves in a 'go get it done' environment. Inevitably, there will come a time (daily in most of the places I've worked) when there's a bug and someone has to do the 'deep dive' to find out details about the problem (one bright QA guy coined the term 'qa forensics') by ssh-ing to the server, finding and watching all available logs while reproducing the problem. Can they search google to get themselves unstuck?

Second, if they mention LINUX or UNIX on their resume, they better damn well know that ls -l is how to get a timestamp of a file. It's worth having a few easy questions about topics that a QA engineer should encounter on a daily (hourly) basis.

The HTTP method question is something you can use to make sure the candidate fully answered the earlier section about performance testing.

SQL Questions There is a person table with columns, id, first_name, last_name, Birthdate
  • Give me the SQL statement to find all of the people named "Chris".
  • How do I find all of the people whose name is "Chris, Christian, Christina, etc"?
  • If you don't know how to do one of the above tasks, how would you find out?

There is another table Address with columns id, street, city, state, zip, person_id

  • How do I find all of the people whose name is Jane and live in San Francisco?
  • How many are there?
  • Walk me through JDBC
  • You have a Java application running against a RDBMS, what are some common optimizations done to enhance the performance of the application at the database level, java level, or interaction in between?
  • If you don't know the answer to one of these questions right now, how would you find out?

Yes these are pretty lightweight questions. Someone who aces this section should be pretty comfortable getting test results from a database. If you're lucky, you can also find someone willing to own the application of db changes to your test environment. The really good candidates will be able to give you a bunch of ways to complete the second tasks: joins, sub queries, etc.

Leadership Questions:
  • Have you lead a testing team? Offshore?
  • How do you motivate a team?
  • Will vs. skill?
  • What communication is important? How do you make this happen?
  • What do you do when the team comes from a letdown?

I would generally only ask these questions when dealing with a team lead candidate or someone going on the manager track.

It's worth noting that I'll usually tell a candidate the answer to a question if they get it wrong or can't answer. It's the interviewer's call about how to deal with this. It doesn't hurt to always sell your company, and a company that supports its team is a company where people want to work.

The interviewer has the option of scoring the candidate as a whole (1-move now, 2-interview after the other 1s, or 3-pass...) or by section. I'll let you come up with your own scoring system.

By the end of this line of questioning, the interviewer should have a pretty decent picture of your candidate, the accuracy of their resume, and most importantly whether it's worth calling them into the office for a face-to-face interview with your team -- which is usually a big investment!

I'll put together notes on the resume-sorting and interview processes for future posts.

12/12/2007  — 

ruby, python, or php?

Indeed Jobs has a neat tool for showing trends in job postings. Seems like python, php, ruby are all on the upswing as expected. Cold Fusion is on the downswing (as expected). Just for kicks I threw QA in the mix. Guess which term has more job postings? Yeah, if you are still trying to figure out what language or framework to learn next, consider "QA". You'll have a larger pool of jobs waiting for you.

04/02/2006  — 

Selenium: Neat!

Selenium is a really promising tool. Can't wait till they get some documentation together.... It's tempting to dive into it and just learn it all...