1 Alyvix basic principles
1.1.1 Critical user transactions
An application transaction is the gap between an application interface interaction and its result visualization. Several factors affect the time that an application takes to fill each gap. They mostly have to deal with software optimization, hardware resources and telecommunications channels.
A desktop application that is fetching a database takes time to retrieve and visualize data. The client software could be poorly optimized, the database could be available from few and slow virtual machines and the network traffic could be congested at that time: in such a scenario end users suffer latencies or even downtimes.
Ensuring the performances of business critical transactions, from the prospective of end user perception, is an important asset for organizations that provide IT services. Monitoring those performances is the first step to improve the quality of service.
1.1.2 Visual Synthetic Monitoring
‘Visual’ means that Alyvix look at an application graphical user interface (GUI) exactly as you do: If you can see, something on your screen Alyvix can too and vice versa. ‘Synthetic’ means that Alyvix synthetizes human users behaving like them: if you can interact with an application GUI, no matter what the interaction mode is (mouse clicks, keystrokes, etc.), Alyvix can too. ‘Monitoring’ means that Alyvix system tracks performances of application transactions: you can analyze time series of your business critical transactions, where latency spikes and downtime gaps are evident.
The idea is to monitor certain user interaction flows where business critical transactions take place in desktop applications or web services. Those flows can be replicated as sequences of transactions: Alyvix provides tools to define every transaction and to serialize transactions within executable test cases. As a result of executing test cases with Alyvix it is possible for every listed transaction to check its availability (i.e. it does not fail) and to measure its responsiveness, so the amount of milliseconds that the application takes from the end of the last interaction to the appearing of the defined graphic elements.
Executing with Alyvix the same test case continuously (e.g. hourly) and from different locations (having several cloned machines sending performance data from several hotspots) it is possible to assess the quality of service of the given application. Alyvix system shows and notifies when (date and time), where (machine IP) and why (application screenshot with graphic elements to detect) that the quality of service decreased or lacked at all. That means doing visual synthetic monitoring.
1.1.3 Alyvix solution
Alyvix is a solution for monitoring critical user transactions of any application from the point of view of the end user visual perception as Alyvix is a software system for visual synthetic monitoring.
Alyvix automates any application, interacting with any GUI exactly as a human would do. Alyvix measures how long application transactions take and visualizes their performances in monitoring systems. Alyvix reports HTML pages containing the details of each test case step.
Alyvix certifies that users are able to successfully complete a certain application task, in a given moment and location, with a certain quality of service. IT operation teams can modulate infrastructure resources. Clients of IT companies can check the SLA with providers.
1.2 Graphic detection and interaction
1.2.1 One transaction, one Alyvix custom keyword, one performance
Alyvix provides tools to define any application transaction. Defining one transaction means building one so-called Alyvix custom keyword, so that Alyvix users have to select graphic elements on the screen and to specify the modes to interact with them. Executing a custom keyword Alyvix starts a process aimed to detect the defined graphics and then to interact with them.
The Alyvix performance measurement system guarantees the output performance time of a transaction is net, so it is provided without taking into account the processing time of detection and interaction. Alyvix can provide one performance for each executed transaction.
Executing a test case means executing a sequence of keywords including Alyvix custom keywords. Each of them defines a transaction and outputs a performance. The latter are strictly calculated from the start of keyword executions to the time when the graphic elements they defined appear on screen. For a custom keyword within a test case, that means from the end of the previous keyword execution to the appearance of the current one.
1.2.2 Alyvix custom keywords: types, components, regions of interest
Alyvix custom keywords can detect any type or amount of graphic element on a computer screen and interact with them in any modes (e.g. mouse clicks, keystrokes). It is possible to build several types of keywords to work with several types of graphics: images, rectangles, text strings.
The Alyvix Finders are GUI tools working on screen that allow you to select an ordered sequence of one main and multiple sub components for every Alyvix custom keyword that you build. Keywords exit successfully if and only if Alyvix properly detects all their defined and listed graphic elements. Right after a successful detection step Alyvix can interact with components as defined, so if some component is missing there cannot be any interaction phase.
The reasons for multiple components in keywords are the following: mainly, it is necessary to disambiguate groups of elements partly identical and secondly, that also allows to interact separately and in different ways with each component. The multiple component selection strengthens test cases, because users are able to define unambiguous transactions.
Users have also to draw the so-called rectangular regions of interest (ROI) for every custom keyword with multiple components. ROI bind the main component of a keyword with its sub components. Alyvix looks for the main component within the entire screen, but it looks for sub components just within their own regions of interest.
Regions of interest allow flexibility to transactions: no matter if some sub components will be rendered in a different position on screen, if they lie into their ROI Alyvix will be able to detect them.
1.2.3 Alyvix Object Finder: multitype components
Besides the basic Alyvix tools (i.e. Image Finder, Rect Finder, Text Finder) it is also provided the Object Finder which is a multitype container of basic Alyvix Finders.
The logic behind Object Finders is the same as basic ones (several components with their regions of interest), the difference is that Object Finder components are basic Alyvix Finders of several types. For example, as main component could be an Image Finder, as first sub component could be a Rect Finder and as second sub component could be a Text Finder. Each sub component binds to the main one through ROI.
Text Finders can take the text string to search as an argument. That means a useful Object Finder is made of an Image Finder as the main component and a Text Finder as a sub component. In this case, users can recall the same keyword with different arguments in order to interact with different elements that are identical on the image side but different on the text side. That is exactly the case of element in a tree view: every element in the list has the same arrow icon on the left and different text descriptions on the right.
1.2.4 Interaction modes: mouse clicks and keystrokes
Users have to define one interaction mode for every component (i.e. main and sub ones) in every Alyvix custom keywords they build. Alyvix can interact with GUI elements in the following ways: doing nothing, moving over the component, left/right/double clicking the component and typing text strings, keyboard shortcuts or a mix of them.
Text strings or keyboard shortcuts can be passed to keywords as arguments: users can reuse the same keyword multiple times within a test case, but each time interacting with different keystrokes. Furthermore, it is sometimes faster just to push keybindings to an application instead of browsing its GUI to complete user transactions that are not critical (e.g. browsing application menus, closing application windows).
1.3 Performance measurement and outputs
1.3.1 Transaction thresholds and performance
It is necessary to define the timeout for every keyword as the amount of seconds that Alyvix waits continuously looking on screen to detect the defined graphic elements of the running keyword.
The performance measurement for each keyword starts immediately after its execution, which equals most of the time (i.e. in the middle of a test case with a sequence of keywords) with the interaction phase end of the previous keyword. When timeout thresholds are exceeded because Alyvix detects anything, keywords fail and break test cases.
For each critical user transactions, one has to enable the Alyvix performance measurement and to set their own performance warning and critical thresholds. That means Alyvix returns three numbers (i.e. amounts of seconds about one performance and two thresholds) for all keywords with enabled performances, so that monitoring systems know all of those transactions that have exceeded their thresholds.
In case of one keyword fails or if one or more keywords pass their thresholds, monitoring systems can show a proper status tag for each test case (e.g. OK, WARNING, CRITICAL, UNKNOWN), give a detailed description about the last successful transaction and send email notifications to promptly alert system administrators.
1.3.2 Command line output and HTML report
The Alyvix deployment provides a command line tool to execute already built test cases (i.e. through Alyvix RIDE, which is the Alyvix test case editor). System integrators can script a command to execute one test case and to get its performances.
The command line output of an Alyvix test case is in the ‘Nagios Plugins’ format. The first row is a text string pipe with the overall exit tag about the test case status, a verbose description about the last successful transaction in case of failed tests and then all the transaction performances (name and seconds) each of them coupled with its warning and critical thresholds. From the second to the last row all transaction performances are listed with their exit status tag, their name and their amount of seconds.
It is also possible to provide an output folder as argument to the Alyvix command line tool in order to save an HTML report, after a test case execution, with the details about all the executed transactions. For each keyword the screenshot of its detection is reported, certifying the screen appearance in that moment: component selections and regions of interest are also drawn. In case of test case failures an animation visually explain which component has not been detected, blinking its region of interest where it should be.
1.3.3 Performance database and Windows Performance Monitor
Alyvix basic keywords allow users to store test case data and performances in a database and to publish them in a CSV file or in Windows Performance Monitor. In the latter way, a generic monitoring system can get that data as any other system statistic (e.g. CPU load, memory usage).
Store Perfdata keyword organizes data coming from test cases in three tables of a single-file SQLite database (one for each test case). In the table runs, every executed keyword takes place as a column: every test case execution generates a new row with a time stamp and a performance for every transaction.
The sorting table reports serial integer numbers (starting from 0) as execution order of keywords, a -1 under a keyword that fails or a NULL in case a keyword has not been executed. In the thresholds table there are the warning and critical thresholds for every executed keyword. Alyvix adds a new row in the latter two tables just in case something is changed from the previous test case execution.
Installing Alyvix (through the Internet) also deploys Alyvix WPM Service by default in the list of available Windows Services (the service runs automatically at boot). Publish Perfdata relies on that service in order to expose all data form the database tables to Windows Performance Monitor. Every test case transaction is a counter that can be added to the ongoing time series chart, but mostly it can be fetched through monitoring agents (e.g. NSClient++) by monitoring systems.