It is critical to choose the right test automation tool if you want a quick return on investment from your automation tests. Yet, selecting the right test automation tool is frequently challenging, especially for applications like Dynamics 365.
The Dynamics applications are somewhat complex web applications that contain concepts such as “configurable elements”, “nested iFrames” and “dynamic locators”. These concepts make it difficult to use conventional test automation tools such as Selenium WebDriver.
In this article, I will discuss different tools that are available for automated testing with Dynamics CRM applications and how at Triad we approach the selection process for ourselves and clients.
What is Microsoft Dynamics 365?
Microsoft Dynamics 365 is a widely used cloud-based business application platform that combines components of Customer relationship management (CRM) and Enterprise resource planning (ERP), along with productivity applications and artificial intelligence tools.
Why test Dynamics 365 applications?
Organisations have different business processes that come in many shapes and sizes. Dynamics 365 is flexible and can be configured according to the organisation’s requirements, yet this flexibility comes with risks that need to be considered, such as:
- Do these new configurations meet the customer’s specifications?
- Have these changes introduced regression issues to the existing system?
- Have we documented the process correctly?
- Does the application still work as expected after Microsoft updates?
Manual testing helps identify issues in new configurations, and automated tests are useful in identifying regression issues.
Testing also helps to enhance the usability of the Dynamics 365 application being configured, which hopefully translates into happy stakeholders all round – from developers to end users.
Why use test automation?
Regression testing is a repetitive task of re-running existing tests to ensure that previously developed and tested applications still perform as expected after a change or an update. It is a very time-consuming process if performed manually.
Test automaton is used to speed-up the regression testing. Automated tests are version controlled, reusable and can be executed anytime as part of CI/CD pipelines or as scheduled jobs. Business readable automation tests written in BDD Gherkin syntax can be used to document application behaviour.
Dynamics 365 test automation challenges
As mentioned above, Dynamics applications are relatively complex web applications that consist of “configurable elements”, “dynamic locators” and “nested iFrames”. It is important to choose an automation tool that can handle such applications and features.
The good news is that there are certainly tools available in the market.
Test automation tools for Dynamics applications
Here are the four best test automation tools that are compatible with Dynamics applications. In no specific order;
- Microsoft EasyRepro
EasyRepro was created by Microsoft to provide Dynamics customers with the ability to facilitate automated UI testing for their organisation. It provides an easy to use set of commands that make setting up UI tests quick and easy. The functionality provided covers the CRM commands that end users would perform on a typical workday, and Microsoft are working to extend that coverage to more functionality.
EasyRepro is built from the open source Selenium WebDriver, which is used widely within the test automation industry. The EasyRepro framework is open source and available on GitHub. It can also execute automation tests in headless browser mode.
EasyRepro supports .NET framework and is available as a NuGet package.
EasyRepro NuGet package
- Executive Automats
Executive Automats is a no-code automated testing suite for Dynamics. It is a cross-platform regression testing tool designed and developed to increase MS Dynamics 365/CRM deployments speed. It also works with other web-based applications such as SharePoint, SAP, Outlook.
Executive Automats contains features for assertions, validations, try-catch, conditional checks, loops, negative testing, scheduler and reporting. It supports on-premise installation and Azure DevOps integrated API.
You can also record user actions and parameterise them to create automation tests. Executive Automats can also be used for performance testing of Dynamics CRM applications.
LEAPWORK is a no-code automation platform that makes it easy to build and maintain test automations. It contains drag and drop components to create automation tests.
The LEAPWORK automation platform uses Selenium as an engine for automating websites and web applications, but all the Selenium code for automating tests is generated ‘under the hood’. The tester does not, at any point, have to worry about reading or writing a single line of code.
The LEAPWORK components (building blocks) can be used to automate form fields of any kind in Dynamics applications.
- Model a system under test as BPMN-style (Business Process Modelling Notation) flowcharts.
- Automatically generate test cases from the model, optimising testing for time and risk.
- Define test data at the model level and generate data at the same time as test cases.
- Export test automation scenarios, automatically generating coverage-focused automation scripts.
- Export test cases to test case management tools such as Jira, ALM, Octane, etc.
- Analyse test results and manage existing artefacts, with visual dashboards and a file management system that introduces traceability between test assets.
Automated tests can be executed across open source, commercial or homegrown frameworks. You can build flowcharts using a range of drag and drop importers and accelerators, and automatically generate optimised test cases.
Test Modeller supports many different programming languages and automation frameworks, covering web apps, mobile apps, API testing, performance testing and database validation.
TestModeller.io also provides a UI Recorder, as an extension to Google chrome, Firefox and Internet explorer.
The UI Recorder automatically captures tester activity on the browser and imports it into Test Modeller. These recorded tests can be executed on Google chrome, Firefox, Microsoft Edge and Safari.
For Dynamics 365 this means creating models that automatically generate C# code, which can be plugged directly into an automation framework for Dynamics 365, such as EasyRepro.
Comparison of tools
The table below provides a comparison of the Dynamics 365 automation tools based on key features:
|Features||Microsoft EasyRepro||Executive Automats||LEAPWORK||TestModeller.io|
|Licence Type||Open source||Paid||Paid||Paid|
|Test development platform||Windows (requires .NET framework)||Cross-platform||Windows||Cross-platform|
|Application under test||Dynamics 365||Dynamics 365, SharePoint, SAP, Outlook and other web applications||Dynamics 365, desktop, Citrix, web and SAP applications||Dynamics 365, web and mobile applications|
|Coding skills required||Yes||No||No||Yes|
|Continuous integration||Popular CI/CD tools (e.g. Jenkins, Azure DevOps)||Azure DevOps (VSTS)||Popular CI/CD tools||Popular CI/CD tools|
|Installation type||Nuget package added to unit test project||SaaS / Cloud, On-premise||Cloud, On-premise||Web-Based / Cloud / SaaS|
|Type of testing||UI functional testing||UI functional and performance testing||UI functional testing||UI functional, API and performance testing|
|Record user actions||No||Yes||Yes||Yes|
It is always important to choose the right tool for the type of application you are automating. As I have shown, there are a number of commercial and open source test automation tools available for Dynamics 365, but their features vary considerably.
Dynamics 365 is a complex application to automate, due to the nature of its configurable elements. However, knowing the automation tool options available for such applications gives you a head start when developing test automation frameworks.
At Triad, we give utmost priority to selecting the most suitable tools for the application type, technical stack and client requirements. The key features we consider when selecting an automation tool for Dynamics 365 are:
- Compatibility of the tool with Dynamics application
- Users of the tool (QA, Dev, PO, etc.)
- Internal and external support for the tool
- Support for running tests in headless browser mode
- Cost of licensing, maintenance and training
- Easy interface to create, maintain and version control tests
- Support of different testing techniques (BDD, TDD, etc)
Automated testing is becoming an increasingly important part of any testing strategy, so it warrants an investment in research and controlled experimentation. Alongside this Dynamics 365’s usage is growing. I’d always recommend investigating several tools thoroughly before proceeding with a project, and we always try to talk to a variety of users who have used a tool before deployment.
Partly this is because different tools will fit with some project types better than others but also because the reality of usability, the human experience if you will, is such a significant contributor to how successful even automated testing tools turn out to be.
Finally, especially with emerging technologies, don’t be afraid to work with trusted third parties to bring expertise and capacity onboard quickly. Triad has over 30 years working with private, public and third sector organisations drawing on experience in the toughest of environments to identify the right solutions for digital transformation. We can provide extra support developing an automated testing strategy or with resources to oversee and manage your testing function.
So, whether it’s advice and guidance, project and product delivery, or additional capacity and expertise, we can find the right people to deliver your technology requirements.
Venu Botla is passionate QA and Test Automation consultant with Triad Group Plc. He has extensive experience developing test automation frameworks to support releases with CI/CD pipelines, implementing QA best practices and leading QA teams within the public, private and third sectors. He has helped organisation to implement QA process in Agile teams, integrating testing at each stage of the software development life cycle and preparing automation test strategy to support CI/CD pipelines. He also has a special interest in making web application more accessible and passionate about learning new testing tools and technologies.