Tuesday, June 6, 2017

2017 OSHG: IMA Test WG: Comparison suites for versions, modules, etc.

IMA Test WG:  
Comparison suites for versions, modules, etc. 
Since we have options in OpenSim, we need comparison data to provide a basis for choice.  Here I describe some of the things we will need to test.  Preparation for the tests will require gathering of collecting a test suite for each measurement objective.


*******************************************************************
  • (More after the break, scroll down!)
********************************************************************** 

Metaverse events, current and upcoming

**********************

Comparison tests

A need for comparing physics engines is already recognized.  Wherever we have alternatives in OpenSim software, we will need comparative data to use in making our choices.   

Time and capacity tests

Some tests used in QA may be used, but specific test scripts will probably have to be prepared to record and report data specifically needed for comparison.  
Some of these tests are likely to focus data on time and capacity.  Such tests will probably be done by running all the software to be compared on the same machine under controlled conditions.  The IMA R&D server will probably be suitable for these tests, although other servers may also be used.  Comparison tests of this kind will probably not require many volunteers.

Performance comparisons, a test suite

By performance comparisons, I mean answers to the question "How well does in match what we expect in a simulation?"  I have seen anecdotal evidence that suggests to me that the .9 version of open simulator supports the operation of rail cars very well.  I would like to be able to test and document that conclusion if it is true.  We all have probably thought of a test we would like to run to compare OS 0.9, OS 0.8, and Halcyon in producing some simulation effort.  

 Performance tests would probably be done with standardized OARs carrying test objects and test worlds. For example, we might have a test OAR with a rail system and rail cars to assess the capability of physics engines to support a working model railroad.  A report on this issue would include videos of the more successful simulations.  

 The OARs and reports would be archived by IMA to make them available for reuse and review.  

Test sources

Initial tests will probably come from developers, grid operators or others wanting information from the tests.  Later, the TestWG may develop some tests in response to a recognized need.  

Tests may originate from observation as in the case of this comment by Lisa Laxton:
"I also tested my inner tube physical scripts... they work great in the test grid where PhysX is the engine with Halcyon... not working well here in outpost alpha ... my physical inner-tubes do not fake behaviors... they are simply made physical and react to the wind while floating slightly in the direction of the seated avatar head... in the test grid you can raise the tubes in the air while in them and they fall to the water and stay buoyant, or you can sink them and they will float back to the top, or you can rotate them to change direction... you cannot do any of that in outpost alpha so that is a major difference between the physics engines and versions of simulator."
From this observation, we could construct a test suite consisting of a standard set of circumstances and desired observations, to go into an OAR for testing physics.

IMA test planning
**********************************************************************************

News and Notes

The Hypergrid WIP Show

  • The Hypergrid WIP is a one hour "show & tell" of works in progress. 
  • Everyone is invited.  Building, scripting, entertainment-- whatever you are working on.
  • Selby may capture video of presentations in voice, for posting on YouTube.
  • The WIP show normally meets on 2 Sundays a month at 12 pm California time.
  • The show meets at the Pandora location on the second Sunday 
  • But may not meet in the summer.
  • And at Cookie II on the fourth Sunday.  
  • To keep up with the WIP meetings, join the Kitely group, Work in progress.

Next WIP meeting

    • Sunday June 11, noon SLT (California) time
    • Pandora Location: (second Sunday of the month)
    • Pandora allows presenters to run high threat OSSL functions.
    • world.narasnook.com:8900
    • Put the line above in your World Map next to Find.  Click Find. TP
    • At Narasnook, use World Map to search for Pandora
    • Cookie II location (fourth Sunday of the month)
    • HG address below: paste into the World Map next to Find. Click Find, TP
    • grid.kitely.com:8002:Cookie II 
    • in Kitely: paste into Nav (top) bar of Firestorm, Enter.
    • hop://grid.kitely.com:8002/Cookie II/68/369/22

            Previous Articles from the WIP show 

            HG links-- depending on your interests 

            Communities in the virtual worlds

            Radio in the virtual worlds

            Metaverse beginner help

            -




            Posted by at
            Labels: , ,

            No comments:

            Post a Comment

            Note: Only a member of this blog may post a comment.

            Subscribe to: Post Comments (Atom)