Accessibility is extremely important to me and to my company, so important that I have strong accessibility standards and practices built directly into my workflow. One of the tenets of that practice is to use both live testing with screen readers and automated testing to look at site-level and application-level trends.
One of my goals for Siteimp was to make it capable of testing anything running over localhost. Once I implemented an accessibility scan, I started using it as part of my automated testing workflow for Siteimp itself.
Circular logic as a development strategy
This was one of those projects I more or less had to complete by myself, because I could never have explained what I was doing without sounding unwell. I needed Siteimp, so I built Siteimp, but I needed Siteimp so badly that I used it to build Siteimp.
Siteimp quickly entered my regular workflow and promptly annoyed me... err, proved its value. My scans were going well. My accessibility scores were high. Then they all fell off a cliff.
This happened at one of those especially rude moments in software development, right near the end of the final sprint before beta. I was polishing the app shell and trying to finish a few visual details before moving on to the monitoring UI. That is exactly the kind of moment where you want to push through, wrap things up, and ship. Instead, my accessibility scores lit up across the app and told me, very clearly, that I had broken something.
The drop was not isolated to one page either. It showed up across the interface, which was a pretty good sign that I had introduced a systemic problem rather than a one-off mistake. That was annoying, but it was also useful. Broad regressions are often easier to diagnose than small, slippery ones because they point toward shared styling, shared structure, or some other common layer.
When a regression catches you at the finish line
With my finger hovering over the delete button as I contemplated sending my accessibility standards into the ether, I decided that no, I should probably just debug it. After all, “when faced with a regression, delete the requirement” would be a pretty lousy learning resource to hand over to the world.
The culprit turned out to be contrast ratio problems I had introduced while working on the titlebar. Once I suspected that area, the fix was straightforward. I plugged my SCSS color variables into my SCSS contrast grid and saw the problem almost immediately.
One quick tweak later and my testing had returned to normal.
Testing the tool on itself
This was a strangely revealing part of building Siteimp. I was not only testing it against another instance of Siteimp running locally, but also against a number of my live websites. It was very hard not to jump back and forth between improving Siteimp and fixing issues on my own sites the moment I found them.
On one hand, that was a great way to validate that the product was doing what I wanted it to do. On the other, it was a little maddening. It is hard not to think about how many hours I have wasted over the course of my life chasing issues manually because I did not yet have the exact tool I wanted.
I should probably have gone all in on the desktop version years earlier. But annoying as this regression was, it also became one of the clearest proofs that Siteimp was doing its job. It found a real accessibility problem, across real pages, before I pushed further toward release.
Conclusion
That is the whole point of a monitoring and reliability workflow. You set standards, measure against them, catch regressions quickly, and fix them before your users pay the price. In this case, the tool happened to catch a regression in itself, which is either a strong endorsement of the product or a very elaborate joke at my expense.
Either way, I will take the result. Siteimp caught the problem, I fixed it quickly, and the scores returned to normal. For a tool built to detect negative change, that is about as honest a test as you can ask for.