Sunday, June 4, 2017

2017 IMA: OSHG: IMA Test WG: suites needed: Quality Assurance

IMA TestWG:  suites needed:  Quality Assurance
The IMA TestWG will be developing test-management methods for several classes of tests.  One prominent class is Quality Assurance.  These tests are done (or should be done) when software under development is released for production use.  QA generally requires a suite of tests designed to identify failures that are not likely to show up in the ordinary testing done during development.  This article suggests the general method that will probably be used
*******************************************************************
  • (More after the break, scroll down!)
********************************************************************** 

Metaverse events, current and upcoming

**********************

What will IMA do?

  • Publicize tests under development and tests needed.
  • Host test materials such as OARs with test objects and tester protocols.
  • Host test software that tracks satisfaction of required test conditions.
  • (Some QA tests may require tests under Windows, Linux, and Mac.)
  • Archive and curate OARS and test protocols. 
  • Archive and curate tests results.
  • Provide public access to curated content.
  • Seek to become the go-to place to find tests useful in OpenSim.

Test suites

In software development, a test suite, less commonly known as a 'validation suite', is a collection of test cases that are intended to be used to test a software program to show that it has some specified set of behaviors. A test suite often contains detailed instructions or goals for each collection of test cases and information on the system configuration to be used during testing. A group of test cases may also contain prerequisite states or steps, and descriptions of the ... tests. -Wikipedia
Note that in virtual worlds a test suite is likely to use specifically designed test objects.  These would be stored in an OAR file to be uploaded and installed as part of the test protocol.

Automatic test suites

An ideal is a test suite is coded so that no human intervention is needed.  The tester simply downloads a program and starts it running.  The program gets whatever files it needs, runs the code to be tested, and reports the results by email or other internet communication.  
In this case, the coder group would write each item in the test suite, specify the sample configurations, and receive the results.  That group might have a script to review and summarize the results or it might do that work by hand.  In any case, the coder group would produce a public report and send it to the test group for archiving.  

The role of the test group in this case would be to recruit testers who can use the specified configurations, follow the test executions to ensure the sample requirements are met, and distribute/archive the results.

Quality Assurance

Quality assurance (QA) is a way of preventing mistakes or defects in manufactured products and avoiding problems when delivering solutions or services to customers; which ISO 9000 defines as "part of quality management focused on providing confidence that quality requirements will be fulfilled".  This defect prevention in quality assurance differs subtly from defect detection and rejection in quality control, and has been referred to as a shift left as it focuses on quality earlier in the process. --Wikipedia

QA: Permissions in OpenSim

There are widely known problems with the retention of permissions settings in OpenSim objects.  The OS Devs have investigated those problems and have made software changes in an effort to correct the problems.  They need tests to determine whether the problems have been solved.  
This need is an example of how the community can and, I think, should contribute effort to the development of OpenSim software.  Most of us cannot contribute code, but we can run tests.  

The OS devs have been developing a test suite for QA on the revised code.  I don't know whether the permissions suite will be fully automated or not.   When the test suite is ready for use, the IMA TestWG will recruit testers and manage the data collection.  The test suite will be hosted on the IMA server, along with a test protocol and configurations (operating system, etc.).  IMA will keep a tally of volunteer commits and completed tests.  This will show what configurations remain to be tested.  

Other needed QA tests

QA Tests are needed for any case where there can be uncertainty about whether the software is functioning as expected.  The IMA Test WG will probably focus first on known defects that are the target of repair efforts.  Specific modules where the WG efforts may be applied are:
  • Viewer
  • Script engines
  • Physics engines
  • Rendering
  • User Interface
  • More (TBD)

Test suites requiring human reports and evaluations

Where human reports or evaluations are included, the query instruments are part of the test and the resulting data are part of the results.  The development of test instruments for human responses is a serious and well-established field.  The TestWG will initially take the responsibility for developing such instruments.

IMA test planning


**********************************************************************************

News and Notes

The Hypergrid WIP Show

  • The Hypergrid WIP is a one hour "show & tell" of works in progress. 
  • Everyone is invited.  Building, scripting, entertainment-- whatever you are working on.
  • Selby may capture video of presentations in voice, for posting on YouTube.
  • The WIP show normally meets on 2 Sundays a month at 12 pm California time.
  • The show meets at the Pandora location on the second Sunday 
  • But may not meet in the summer.
  • And at Cookie II on the fourth Sunday.  
  • To keep up with the WIP meetings, join the Kitely group, Work in progress.

Next WIP meeting

    • Sunday June 11, noon SLT (California) time
    • Pandora Location: (second Sunday of the month)
    • Pandora allows presenters to run high threat OSSL functions.
    • world.narasnook.com:8900
    • Put the line above in your World Map next to Find.  Click Find. TP
    • At Narasnook, use World Map to search for Pandora
    • Cookie II location (fourth Sunday of the month)
    • HG address below: paste into the World Map next to Find. Click Find, TP
    • grid.kitely.com:8002:Cookie II 
    • in Kitely: paste into Nav (top) bar of Firestorm, Enter.
    • hop://grid.kitely.com:8002/Cookie II/68/369/22

            Previous Articles from the WIP show 

            HG links-- depending on your interests 

            Communities in the virtual worlds

            Radio in the virtual worlds

            Metaverse beginner help

            -




            Posted by at
            Labels: ,

            No comments:

            Post a Comment

            Note: Only a member of this blog may post a comment.

            Subscribe to: Post Comments (Atom)