As an Appian project professional, you face continuous changes in requirements and process improvements.
You are aware that Appian is a comprehensive, unified platform, rich in solutions, and highly agile. As the projects enter into the corrective maintenance phase, the number of small changes increases, impacting any part of the application. Therefore:
EVERYTHING IS SUBJECT TO CHANGE, even those parts that were considered stable.
Regression testing covers the entire application. Regression is more extensive than the project creation phase testing which focus solely on the new parts. Consequently, if tests are not automated, you may have noticed that regression testing becomes the new bottleneck for delivery. All eyes are on the QA team. Once development is done, deployment has to wait until all regression is passed, which can be cumbersome, if the project has a relevant size.
In other technologies, developers are not as quick as in Appian, and development and test stages are more aligned in time. However, in Appian, changes are implemented very fast, so... what about testing? The key lies on the regression strategy !
Managing regression undoubtedly impacts efficiency, profitability, and ultimately the success of the project, thereby influencing client retention.
What are the most common adopted measures?
If you are automating tests, you will need to involve high-level experts, such as a QA engineers, to change the existing automations. Typically, these individuals have a highly technical not focused on the business side. We usually don't need them full time, so their constant engagement becomes expensive, as are difficult to share among several projects (each project has its own timing!).
If you are not automating, you need to engage manual testers. However, since you need to perform regression testing on the entire application, rather than testing a particular new development, you require more testers to complete the tests on time, or the regression will take significantly longer.
It is a Must a good level of test automation for a proper QA management.
When automation is not properly managed, we end up finding than application changes are faster or similar to testing changes. So then...
- Why should I continue automating tests?
- How can I manage the impact of changes on my automated tests?
In our wide Appian experience...
Based on our field experience, we can say that we have seen how every Appian project that exceeds 6 months requires implementing automated tests to minimize deviations (in terms of time and costs). So, what would be the most important points to consider when implementing test automation?
- Carefully choose the test automation tool.
- Understand the current phase of your project.
- Evaluate the level of changes you are facing and the excellence of the results.
1 Test automation tool
You may be aware that there are increasingly powerful tools, cross-technology, claiming to be effective automating the regression. For more information on the alternative options available in the market, we recommend reading other articles on this topic in our blog.
However, its "power" (especially the more expensive the tools are) in the context of low-code, requires some considerations:
- These tools are aimed for QA specialists. Consequently, their level of collaboration is not high.
- They bring notable technical capabilities for generic problems (image recognition, text processing, etc.) but are not focused on a particular platform.
- They are designed for stable projects with minimal maintenance needs.
- The reusability is usually defined at interface level, not at component level, as Appian does.
We would like to provide some significant examples based on our real-world experience, as we have encountered several situations where such automations become more of an issue than a solution.
Appian's Focus on Maintainability
On more than one occasion, we have witnessed test managers from outside low-code technologies, implementing their automation tools in Appian projects.
After investing significant technical effort in automation, the continuous changes in development make their efforts futile, finding out unsustainable to maintain the costs of licensing and QA engineering.
Appian Version Changes
Appian evolves its platform with four releases a year. Version changes in the platform can impact object rendering.
It should be noted that most generalist tools rely heavily on rendering, screen positioning and image recognition (rather than understanding Appian objects).
We recently encountered a major project automated with a powerful and expensive tool, that after an Appian version upgrade failed to run any test. Appian slightly changed how calendars look like, impacting the whole application look and feel. A tremendous testing effort was required to adapt the existing tests since they had become unusable, raising doubts about the real usefulness of the tool in the project.
QA Engineering Specialist
It's no secret that QA automation specialists are highly sought-after and expensive profiles in the market. Therefore, obtaining benefits in a scenario of continuous small changes becomes quite challenging. The more complex the tool, the more expensive the profile, and the higher the project maintenance costs become.
Specialised Tool for Specialised Platforms
Here's where we come into play!
Our accumulated experience in significant Appian projects, both in development and test automation, has led us to create a solution which engages Appian designers, testers, managers, product owners, and functional specialists. In fact, our solution, Appian Automated Testing (AAT), can prove beneficial for every project stakeholder as it doesn't require technical skills or involve code. It is user-oriented and provides a full functional view.
AAT has been highly appreciated by our clients as every team member can understand and maintain the tests.
It has the following key characteristics:
- AAT is a specific tool focused on Appian.
- Programming skills are not necessary; any user, such as a manual tester, can use it and create automated tests in no time.
- It is behaviour-driven rather than relying on other recognition techniques (as discussed in previous blog articles) and can react highly responsively to changes.
- With its reusable components, modifications that impact dozens of test cases can be made in just a few minutes.
- It is collaborative, allowing all profiles to contribute to the testing cycle. Furthermore, it enables any participant to execute or even modify any test quickly and easily.
2 Project Phases
You must align your testing strategy with your project phase. Having a clear understanding of the project phases allows you to dynamically adapt your testing strategy.
New Project
New projects have a minimum regression, but it will increase quickly. We recommend laying the foundations of the project choosing a BDD methodology and a tool that fully supports it, such as AAT.
Project in a Changing Phase
A phase change is an excellent moment to prepare for a next stage. Keep in mind that unless you have been using BDD from the very beginning and you got almost everything automated, you need to seriously consider regression.
Failing to consider regression effort will certainly result in significant deviations.
Project is ringing a bell
If you notice symptoms like late incident detection, UATs taking longer, or lack of test data, brace yourself, you're experimenting regression issues.
There are different ways to redirect testing. Ask us for more information.
Project in a Critical State
If the team is new to the project and hasn't fully embraced it yet, or if the project started poorly and has consistently faced deviations, it will not get better unless you cover all areas, including the regression.
This is the time to seek the advice of one specialist. We are available for second opinions, to help you with strategies and to support you in getting the situation back on track.
3 Evaluate the Level of Changes You're Facing and the Excellence of Results
If the project is highly stable, there isn't much to say... But if there are changes, consider the keys to excellence:
- BDD: Create automated tests during development, don't wait for development to be completed, so that designers can use them.
- Shift-left: Testing should be performed continuously and as early as possible. Run all tests nightly and check the results in the daily.
- Maintain alignment with Appian: 1 Appian component = 1 test component. This will improve test maintainability.
- Run tests in all non-production environments. Yes, in all of them. Is good to know how many things are not working on DEV.
- Conduct regular audits of your code quality.
Conclusion
We could continue discussing in-depth our experience for several more pages, it is our passion at the end!
However, to summarize:
- In most Appian projects, it is beneficial to automate regression, but always with a good strategy and appropriate tools.
- QA should always align with the Agile approach provided by the Appian platform. Losing alignment is one of the main reasons why automation fails.
- The benefits of regression automation are manifold: it helps increase confidence in the project, reduces the impact of staff turnover, energizes and accelerates sprints, and reinforces the value of deliveries.
Every project is different, every situation is particular. Feel free to contact us for a detailed conversation.
Usually a one-week, non-binding proof of concept is sufficient to identify important elements and evaluate the feasibility of solutions.