My Experience with Open Source Testing Tools - 2
Maturity has different meanings when it is related to software testing tool.
First criteria is reliability of the tool.This means tool should generate the results in most consistent and acurate manner.Here Consistency means if tool pass(or fail) a test then it should always pass(or fail) that test irrespective of number of iterations of test suite execution.
Second criteria is the number of features this tool helps in for testing.Again these features vary depending on the type of testing we are talking about.For ex,if we are talking about performance testing,the number of protocols this particular tool is supporting as performance testing mainly deals with triggering transactions between different components having their own protocols.If the underlying testing is functional web testing then features could be like number of browsers(IE,Firefox,Opera etc) it is supporting as well number of platforms(windows,linux,mac etc) tool is supporting.
Third Criteria is activity under that tool.This mainly tells the number of users using that tool as well as the number of people who are working for that tool to fix the bugs and to enhance the features of that tool.This criteria is very much important in selecting a tool.One can very easily get details pertaining to activy of that tool in tool's website.
Open source Community is anouncing that their products are having high quality than the equivalent commercial software.Opponents of open source arguing just the opposite saying that open source software is highly inconsistent and unreliable.Anyway i dont want to extend this with my own arguments.I am trying to tell the advantages one can unleash by using open source testing tool to a company as well as to an individual.
whenever i evaluate an open source tool, i will follow above criteria.I've selected selenium by following the same criteria.
The main reason for people using the open source tools is not whether their need is served, since in most cases you will find a tool to do the job, but whether the tool is mature enough to invest in. Test automation is a major undertaking that has high costs in terms of upskilling test team and creating test scripts, etc., and people need to know that the tool is a good investment, whether or not there is a license fee to pay. People need to look about if the tool is going to be around in five years' time, what levels of support are available, how good is the feature innovation, bug fixing, release schedules, and so on.
Labels: automation, evaluation, functional, maturity, opensource, performance, selenium, testing
