Microsoft Dynamics 365 Regression Suite Automation Tool (RSAT)
Welcome
Guest Blogger Latha Kalaga
Hello World!...for the
first time ever, I have a guest blogger. Please help me welcome Latha Kalaga. I
have been working with Latha for nearly 13 years and she brings a myriad of
expertise on all things technical related to Dynamics 365. Because Microsoft
has introduced her in more than one session as the "number one user of
RSAT", I thought, hey why not get her to write this blog about RSAT. I can
honestly say, Latha taught me everything I know about RSAT, and I hope you
enjoy this post.
Introduction
to the RSAT
Microsoft introduced
the concept of continuous update in July of 2018 to ensure that all Dynamics
365 for finance and operations will be on One Version of the software. The
customer will have 7 calendar days to validate an update that has been applied
to a sandbox environment prior to rolling out the update to production. To
support this activity, Microsoft also rolled out Regression suite automation
tool (RSAT) to help the customers validate the update.
It is very important
to test every update with your mission critical business processes in mind.
Typical manual testing of major critical processes could take 40 to 60 hrs or
more and depending on the number of test cases, a company may have to allocate one
or more resources to this testing effort prior to every update. The cost of
this manual testing prior to taking the update is too high and prone to errors.
The regression suite automation tool significantly reduces time and cost of
user acceptance testing by allowing the acceptance testing to be mostly
automated.
The regression suite
automation tool allows functional power users to record core business processes
using the Dynamics for Finance and Operations task recorder and convert these
task recorders into test suites without having to write a single line of code.
Technical
Details
Regression testing is
an art as much as it is a science. Laying out what you want to test, the
combinations of a process that you want to test needs to happen before any
testing or testing suite finalization comes into play. You will want to start
with a release validation plan layout. This is a paper and pencil exercise in
which the functional testing subject matter expert lay outs the process and the
intended validations of the expected values. It is also important to think
about data dependent testing suites versus a data independent testing suite.
Once the release validation plan is in place, the testing suites can be
assembled from the processes that have been laid out in the business process
modeler.
The business process
modeler that resides in the Life Cycle Services (LCS) is one of the, if not the
most important component of a regression testing life cycle. It is important to
lay out the business processes that are needed to successfully run a business
and then record these business processes using the task recorder provided. For
a task recording to become a regression testable recording there are two
components that are required.
- Validations
- Reference variables
The first component is
adding validations and the second, optional but a valuable component is the
reference variable. Validations in a task recorder allows RSAT to compare the
actual value to the expected value. The reference variable allows the tasks in
a test suite to be chained.
The business process
modeler is synced over to Azure DevOps in two passes. The first pass syncs the
business process modeler nodes as epics and the second sync, synchronizes the
task recordings as test cases over to Azure DevOps. Using the release validation
plan the user can create a test plan in Azure DevOps and add the test cases to
the testing suite. A Microsoft Visual Studio Enterprise level license is
required to create a test plan in Azure DevOps or per user per month license
can be purchased to allow the user to create these test plans.
Once the test suite is
created the test suite can be loaded into the RSAT tool that is typically
installed on an Azure Server or a laptop of a quality assurance tester. The
RSAT tool runs the business processes through the user interface of Dynamics
365 for Finance and Operations as it runs through the steps of the task
recorder while validating the values called for in the task recorder.
The results of the run
are stored in Azure DevOps allowing the user to evaluate the results of each
run and compare different runs of the same test suite. Errors are logged in
Azure DevOps in easily decipherable comments allowing the functional experts and
the user acceptance testing teams to figure out the issues if any caused by the
uptake of an update be it a customization or a Microsoft update. This triage
will enable businesses to identify issues prior to taking the update in
production. The values used by the RSAT tool are decoupled from the original
recording and can be changed in the excel spread sheet that is generated by the
RSAT tool using the task recording. The power of using reference variables to
chain tests allows for running of test suites that are complete end to end
processes without having any manual intervention.
The following is a
screen capture of the type of test suites that can be created. This is a data
independent suite for order to cash that can be run against any dynamics for
finance and operations environment. The inputs and expected values are the only
pieces that needs to be checked in the excel spread sheet that is attached to
the test case prior to running this against any environment.
Below is a screen
capture of the suite from Azure DevOps loaded into the RSAT tool:
RSM
value
At RSM we pride
ourselves on being the First Choice Advisor to our customers. We participated
in the beta release of the Regression Suite Automation Tool and are heavily
engaged in helping improve the tool and the components that go into making RSAT
truly a good testing tool. We are working with several customers to put
together their release validation plan and help them build out their suites
using the tasks recorded and attached to the business process modeler. We also
train our customers in using RSAT if they choose to build out their own suites.
We advocate best practices for all the components starting from what it takes
to create a good business process modeler, a good task recorder that will then
lend itself to creating a powerful RSAT suite. We are participating in the
release validation program that Microsoft announced by enrolling our RSAT
suites that we have built out for our customizations sold through the Market
place. We are one of the leading RSAT users if not the leading user with a staff
that is trained in everything RSAT starting from installation, to test creation
to execution.
Here are some
statistics from our efforts at some of our customers.
- Recorded 140 tests (4 resources, 1 month), 50 processes covered enabled the client to cut down manual testing by 52% (Once we record the mobile processes this percentage will reduce further)
- Recorded 208 tests (4 resources, 1 month), this suite runs in about 2.5 hours. Manual testing takes about 40 hours today for this same set of tests
Conclusion
RSAT has a lot to
offer and it is very hard to capture all the information in a blog post. Please
feel free to reach out to RSM if you would like for us to demo the capabilities
of RSAT or in general the planning and execution of a test suite.
Quote
“Why do we never have time to do it right, but always
have time to do it over?” - Anonymous
References
Again, a HUGE THANK YOU to Latha Kalaga for writing this post. You can connect with her on LinkedIn here.
If you would like to
see me live presenting on the RSAT be sure to sign up for the Dynamics 365 User
Group Conference here.
My session (along with many others) will be recorded and available to attendees
who have registered to receive the recordings. Don't miss out! This is a
special add-on to your conference pass for just $100 that adds a lot of value
that keeps on giving even long after the conference is over.
Comments
Post a Comment