Software Testers Welcome Pack

1. The start

Hi software tester,

we do have a general starting page for all software testers.

You will find it under

Software Current Tests - Software/CurrentTest

This page contains information about the actual test and patches as well as general links to the documentation, FAQ, the test team members and the closed tests. Further more you will find information about the state of the test server (online, maintenance mode, etc.).

On the beginning of the page you will find a collection of links need by any software tester. As the link to the test server, the test server management system and basic help e.g. How to create an account. This page is maintained and kept up to date by the test team leader and the software assessors.

Detailed information about bugs and patches will be presented in the bug reporting system Mantis bugs.cacert.org. Additionally more information can be given in extra wiki pages to give background information about a bug/patch. The links (if there is an addtl. info page available) can be found in the remark field for the bug.

The test starting page gives a list of the test servers that are able to use (at present only one server is active).

For each test server you will find a detailed list of patches that are added on top of the production server.

For each patch you will find 6 entries.
First of all there is the bugs.cacert.org bug number which represents the link to the bug reporting system. This is the source for all information about the tested patch and the place for the reporting (.1). Each report is added as additional note to the bug report. The software assessors review the given report and if there are not enough results the test status in kept on. The main line in the testers portal presents also 2 addtl. infos:

  1. current state of the patch (column 2)
    1. Patch - patch is in full testing state
    2. {-} Patch - patch is rejected, placed on hold, awaits an update, so current testing for this patch set to hold
    3. {0} Patch - patch has been tested and awaits 2nd review by a Software-Assessor and transfer to critical system, addtl. tests can be made, but if there are other untested patches, so give them the priority
    4. {b} Patch - patch has been tested and is under review or transfer to the critical system, no further tests required, patch will be removed soon from the list

  2. column 3: (Update 2011-09-27) {*}

    1. the last update added to testserver - this may sometimes happen. The patch has been tested, but another bug has been found and has been fixed. Now this patch needs new test series

    2. {*} marks the difficulty of the test. 1 star {*} is an easy patch / test, 5 stars {*} {*} {*} {*} {*} (the max) is difficult to test. needs an advanced testing scenario, several test rounds, special permission requirements or special settings that needs to be prepared first, before the tester can start with the tests

For each bug the developer/maintainer is given (.2). He will be the person next to the test team leader to give detailed information on the bug/patch.

A short description of the bug/patch (.3) as it is given in the bug system.

The area where the bug/patch is located (.4) e.g. assure someone, organization assurance. This will help to find the entry point for the testing.

For some bugs/patches some precondition are needed (.5). These will be here e.g. for some test the testing account needs to have Support Engineer (SE) status. Therefore the Admin flag needs to be added to the account. How to do that, later.

Under Remarks (.6) there are additional hints to the patch. Either what to test, where to find the patch or other information.

Sometimes there will be additional patches. To keep the status the latest patch is indicated by (!) . If a patch shows up some errors in the testing and these errors where resolved in the meanwhile by a renewed patch the added update will have the (!) to indicate that it is an update patch.

As the test environment is still under development there are some bugs which will stay there quite a while. For example the Bugs # 835 which will stay at least some time. This bug is treated as a hint that all certificate actions are out of order. The function create clients certificate, create server certificate and the certificate login are not working as long as the "signer implementation" is not established to the test server. This also means there will be no test covering anything to do with certificates. The signer has been activated in the meanwhile, but a CATS testserver is not yet activated. CATS passed flag needs to be set thru the Testserver-Management-System.

What to test?

As long there are no information given in the bug there are no preconditions to keep in mind just try out with different browser, different system settings (e.g. Points, flags etc.).

Basically the testing follows these steps.

  1. Settings for the test environment
    1. Create test account bare preconditions in mind
    2. Insert the required settings to the account (points, flags)
  2. Start the testing
  3. Report your results

In general the productive system is the reference system for the test environment. Do things work on the production system that are not working on the test server? If yes there is a problem with the patch. Check if it is already reported, if not add new bug report. Some things cannot be cross checked with the production system as you might not have the privileges on the production system e.g. for organization assurances. (May be it is a good idea for the Software-Assessment Project Team to implement a reference server as software clone).

Yours

Testteam Teamleader


What is next ?

Software Testteam Welcome Pack

Software/TestTeam/WelcomePack/01-TestersEntryPage (last edited 2011-10-05 23:02:13 by UlrichSchroeter)