Why is it still so difficult?

There are many researched and documented approaches to apply artificial intelligence techniques to testing. So why don’t we have AI automatic testing software already?

Probably it’s the size and irregularity of the problem set, and, I guess, this is the challenge of mimicking the wonder of the human brain. How do you build an application that will figure out how to test a unique program? The program might have varying purpose, structure and size in each instance. It might have different combinations of components and infinite viable execution paths. How do we as software testers approach this problem?

Often we use models of our software to conceptualise it and make it easier to digest. Then we break it up into components and might test each component and its combinations. Or we might approach the system from the aspect of its functionality and try to ensure the expected functions work as they should.

We use certain heuristics to impart value to each aspect  of our systems and then test the ones we feel are most valuable or most likely to fail first. We might include information from different sources including defect registers, product management and knowledge of support calls to impart priority. We use experience and trial and error to determine how to test and what to test.

Then we need to design a method of testing the software. Will we manually test, automate or both? In each case, how can we interact with the software in the most realistic and valuable way? This kind of organic investigation and decision making, and the number of decisions that have to be made as well as the knowledge store required to make them, make it difficult to transfer the entire process in one go to any one automated system (developers, I hope you will appreciate your testers a bit more after reading this 😉 ).

Although test automation techniques could give us clues as to how to model the test process and test activities, these still require a large amount of human input. This input is what we are trying to substitute by applying AI to testing.