15 May Top 5 Killer Pitfalls of Automation Frameworks
Designing and implementing a robust automation framework should be the goal of any test automation effort. The days of each engineer doing their own thing, using their own techniques and methodologies within the same organization are long in the past. This plan – or in reality, lack of a plan – has been proven to end in frustration, missed goals and ultimately a lot of shelf-ware. But designing an automation framework that successfully supports your testing effort is full of pitfalls. Let’s take a look at a few of the most common pitfalls with the hope of avoiding falling prey to them.
Not looking at the big picture
Take time to learn and understand your organization’s overall QA goals and expectations, not just related to test automation. The framework you design has to target these goals and meet the expectations. If you feel the expectations for the automation effort are out of sync with reality then bring up your concerns before getting too far in the planning process. Managing expectations is one of the keys to success. Not understanding the big picture can lead you in the wrong direction from the very beginning.
Not knowing your customer
Your customer in terms of a framework is who will be using the framework to create the automated scripts. The framework is not the script, it executes the scripts via a driver script, function libraries, etc. (the back end) and it dictates the format and design of the automated scripts (the front end). If your audience is more technical in nature and understands object classes and properties and good automation methodology then you have more flexibility in the design of the front end and can give them more control in terms of technical design detail when creating the script. If they are less technical then that will limit the design options of the front end. You will need to keep it as simple as possible and keep the complexity on the back end which will handle most, if not all of the object identification and interaction, etc.
Not knowing the applications (AUTs) in scope
Before beginning the frame design, know the AUTs in scope. Are they process intensive or data intensive or both? This will help you determine the type of framework to design. (Framework types are not covered in this blog) Do they integrate or are they independent? If they integrate then you will need to design the framework to allow for multiple applications to be included in a single script. If they are browser based then maybe multiple browsers will need to be open simultaneously and you may need to account for executing against multiple browser types. These are critical technical design decisions all dictated by the AUTs in scope and easily missed in the design plan.
Choosing the wrong resources
Choosing the wrong resources can have a long-term impact on the success of a framework. There are several options when putting together the engineering team. You can use all external consultants, all in-house resources or a combination of the two. The key is to have the right level of expertise and experience on the team. That may require utilizing some external resources during the most technical phases of design and development to ensure the framework meets the goals, is robust, flexible, scalable and easy to maintain. If external resources are used then it is very important that the remaining in-house resources understand all aspects of the framework in order to build on it and maintain it. Without that knowledge, even a well-developed framework can succumb to the burden of maintenance.
Failing to track, review and report progress
In the early stages of framework development, it can be challenging to capture metrics and track progress. But not doing so leaves wide open the assumption that no progress is being made. Be sure to put in your design a plan to track and report metrics at all stages of development. As part of this process be sure to measure whether or not the framework is meeting your expectations and meeting the needs of the overall project. In other words, answer the question, “Are we headed in the right direction?” Be willing to change course if needed. Flexibility at all stages of development is an important key to designing a successful framework.
