How to be a Good Bug Buster (beta-tester)

What? Why?

If you think that searching for bugs is simply running an application to see if it launches, then you need to read this how-to.

Having programming knowledge can be very helpful, as it allows you to understand what common issues might arise.

First of all, merely launching the application to check for segfaults doesn't provide any useful information; it's more like a challenge to discover how it can fail.

However, there are some techniques you can use to uncover these hidden bugs, so let me share some tips:

Environment

Not everyone works in the same environment or uses the same options. For instance, if a developer works with feature "A" of an application but never uses "B", it is very likely that there will be more bugs in "B" than in "A". Similarly, trying less interesting options will yield similar results.

Try to imagine how many different environments and work methods can exist. For example, if you are beta-testing Enlightenment 17 and want to see if a window minimizes correctly, consider all the possibilities. You can use the mouse to click the minimize button, or you can press Ctrl + Alt + I on the keyboard to achieve the same result. Additionally, think of other factors related to minimizing an application, for example from ibar/engage it can also be triggered the minimization.

Unimaginable Possibilities

Are you creative? You need to be. Think about it: in the previous example, we have seen different ways to trigger the same action, but not all the possibilities are listed here. You need to be creative (or at least familiarize yourself with the application/code) to imagine how many different outcomes can occur. For example:

• Can the window be minimized when it is in maximized mode?
• If another window is on top of one, can we still minimize it?
• What happens if I try to minimize during an e17 restart?
• Which window is minimized by default if I switch to a different desktop and don't manually focus on any?
• What happens if the minimizer module is not loaded?

The previous list is just an example, and you may find it useful or not for testing. It all depends on you. The important thing is to try everything that comes to your mind that makes sense.

Expect the unexpectable

Biggest Headaches

If you are having the biggest headaches trying to catch a bug and still haven't found it, it may be because you are not looking in the correct place. Here are a few examples:

• Warning with async, threads, and parallel tasks: If you have things that run in parallel, nothing ensures that A will happen before B, even if it is declared that way in the code. For example: You have a code that generates a temporary HTML file with statistics, then you open it in a browser in the background to visualize it. After that, you copy it to a backup directory, and finally, since this is a temporary file, you simply delete it. In this example, you could experience a failure with the browser. Why could this happen? I used the example of a browser because it is bloated with unnecessary features, causing it to take some time to open. In the meantime, the script has continued and removed the file, so when the browser is ready to display it, the file is gone.

Combinations

You can more easily find bugs by trying to combine different elements. For example:

• What happens if you run the application multiple times?
• Is the performance affected by running it multiple times?
• Try A while you are doing B.
• Conduct the same tests with different settings.
• Combine parameters/features simultaneously.

Frequent Problems in Applications

This is more like a list of common novice programming issues. If you encounter them, you can hold the author of the application accountable. ;)

• Your system is multi-user, and many applications use /tmp to store temporary data (like /tmp/app1). If another user runs the same application, it will use the same location, which can lead to significant, annoying, and even hard-to-diagnose problems. The most common issues are permission denied reports.
• Multiple instances: Sometimes an application is not designed to be run multiple times. This needs to be managed correctly by the programmer to avoid conflicts between different instances of the same application. If the application is intended to run only one instance, it should prevent another run with a message like application is already running.

Betatest yourself

It is very common for us to report incorrect bugs. This happens when the cause of the problem is not what you think, but rather a side effect of something else. A similar issue occurs when you report that the cause is A, but in fact, it lies elsewhere. Just make sure to identify the real cause of the problem before reporting it.

Other different types of bugs

Pebkac

(Problem Exists Between Keyboard And Chair): After reprimanding the user of your application for being foolish, you must also reflect on your own shortcomings. Pebkac issues are really bugs that you need to fix... yes, you, the developer :), this is known as an usability issue. You may think that users are foolish, but that won't change anything; what can change is your application. Make it more user-friendly, lazy dev!

Wording

A grammar error is also an error worth reporting. If you encounter incorrect messages, such as those written by the tarzanic of Thanatermesis, you must report them as well. Update: Since version Topaz, Elive's translation tool includes the option to translate into English, meaning you can translate from tarzanic (the application's original language) to English!

The process of beta-testing wording doesn't end here. An interface must use optimal messages. In short, this means:

• avoid overwhelming users with excessive information (they don't read it, and they realize it)
• provide clear and direct messages, without potential for misinterpretation
• keep messages concise
• ensure messages are easy to understand, but not so simplistic that they alter the logic and meaning

User Experience (UX)

This is an entire science that requires knowledge of what constitutes a good interface. While much information exists on this topic, you can still evaluate these aspects because you are a logical and coherent person. The interfaces of applications need to be coherent and logical, easy to use, handy, and user-friendly. If you believe that the interface could be improved in a different way, please report it with details on how it should be designed.

This involves:

• User Interface Design (UI)
• Optimal Positioning of Elements (buttons, text, etc.)
• Feelings Evoked by the Interface, Its Use, and Actions
• Usability
• Information Architecture (IA)
• Handy Application, Achieving Efficient and Desired Results
• Helpful Tips (ways to assist the user if an element is difficult)
• Avoiding Overload of Elements/Text/Images/Tips/Messages/etc.
• Interaction Design (IxD)
• Psychology

Remember

It is very important to report information to the developers; problems don't solve themselves, so please take the time to do this. You are the true end beneficiary of a well-functioning application, not the developer.

Think as if you were the developer. First of all, you need to be able to reproduce the problem, so you must provide the developer with the necessary information to replicate it (do it yourself and write down the steps). Finally, include all relevant information that the developer may need. Make their job easier, and you'll increase the chances of having the issue fixed!

Before you report any bug, try to have someone else reproduce it. This can confirm the bug and verify that it can be correctly replicated. You can ask any volunteer in the IRC chat channel.

Motivational end sentence: Girls don't really like muscles; they prefer intelligent guys. Smart beta testers who catch lots of bugs are very intelligent individuals.

You don't belive me ? ​see it yourself with your eyes, really... ​it is that way