Forward to this list for filing purposes and to ensure whole group has the material.
-------- Original Message -------- Subject: Fw: Evaluation Date: Thu, 15 May 2008 21:16:22 +0100 From: Graham Taylor graham@openforumeurope.org To: Shane M. Coughlan coughlan@fsfeurope.org, Georg C. F. Greve greve@fsfeurope.org, Rod Norman rod@openforumeurope.org, Bob Blatchford bob@openforumeurope.org
Just received G ----- Original Message ----- *From:* Peter Henderson mailto:p.henderson@ecs.soton.ac.uk *To:* 'Graham Taylor' mailto:graham@openforumeurope.org *Sent:* Thursday, May 15, 2008 4:51 PM *Subject:* RE: Evaluation
Dear Graham,
Attached is a draft evaluation of Certified Open, performed by one of my colleagues in OMII. We found the assessment of two OMII software components quite enlightening. This draft is for your comments. We will not publish without your agreement and ask you not to circulate, cite or quote until we at OMII have decided whether or not to proceed with publication. I will, however, let Stephen Rowe have sight of it (with the same constraints). Stephen and I met on April 22^nd and agreed that dstl would seek to determine how they could evaluate your offering. Basically, they need to find an appropriate person. By sending our report to Stephen, I trust this will prompt him that (as far as I am aware) this intention is still outstanding. Apart from this publication to dstl, I will withhold publication until I have your endorsement. I look forward to hearing from you
Regards, Peter
---
Professor Peter Henderson
Open Middleware Infrastructure Institute
School of Electronics and Computer Science
University of Southampton, SO17 1BJ
+44 (0) 23 8059 3440
*From:* Graham Taylor [mailto:graham@openforumeurope.org] *Sent:* 23 April 2008 12:03 *To:* Peter Henderson *Cc:* Rowe Stephen *Subject:* Re: Evaluation
Peter, thanks for this and thanks again for the meeting. I was very encouraged by the almost identical language we were both using and the commonality of the objectives.
Firstly on Certified Open I can confirm the trial will remain open for the forseeable future. As discussed we will only be in a position to formally launch once we are satisfied we have a balance between industry and user buy-in. Equally that all the lessons from trialling have been absorbed. I now dont expect this to be reached before end June. So your OMII input will be highly valuable.
On the wider discussion with Steve and MOD I will of course be very happy to present CO to the wider supplier group if you think that would be helpful, and possibly ease any concerns they might have. I would confirm that during the trial period all the data is entirely confidential, and there is no cost involved in the exercise. I have just received one input back from one of the triallists on the LG trial which suggested a 30 min allowance for completion of the online form - so it is not an undue effort. We would also be very interested in a discussion whereby MOD were provided full and confidential access, or rights to maintain an independent version.
Finally, by coincidence I bumped into Andrew Boswell in Brussels last week - we were both on the same Eurostar. He asked me to send his regards to you. He is now working for himself as a top level advisor, mainly in the telecomms/network space, and is currently on a long term assignment for Ericsson.
Best Rgds
Graham
----- Original Message -----
*From:* Peter Henderson mailto:p.henderson@ecs.soton.ac.uk
*To:* 'Graham Taylor' mailto:graham@openforumeurope.org
*Sent:* Friday, April 18, 2008 4:11 PM
*Subject:* Evaluation
Dear Graham,
As promised, I have now assigned someone to evaluate the Certified Open tool, by entering data on some of the OMII components. This guy wont be able to do the work until the beginning of May. Im assuming itll take him about a week (by the time he has reported back to me). Can I assume that the free period will still be available to us, even in May? Once we have evaluated it, Ill report back to you. You may want to come down and talk to us about it. And, of course, Ill tell Steve what weve done. Independently of that, Ill see if Steve wants to do something directly. I expect to see Steve next week
Cheers, Peter
---
Professor Peter Henderson
Open Middleware Infrastructure Institute
School of Electronics and Computer Science
University of Southampton, SO17 1BJ
+44 (0) 23 8059 3440
*From:* Graham Taylor [mailto:graham@openforumeurope.org] *Sent:* 11 March 2008 20:12 *To:* p.henderson@ecs.soton.ac.uk *Cc:* Rowe Stephen *Subject:* call
Peter, it was very good to speak to you this afternoon. It never ceases to amaze me at the extent at which ICL has managed to percolate the ecosystem in ICT.
As promised I've attached the brief user trial spec that we have prepared for Certified Open. I look forward to meeting you on the 8th at about 1530. I'm going to be in France from friday until the 7th but if you want to speak to me either email or you can phone direct on +33 5 53 90 10.
Best Rgds
Graham Taylor
Dear all
I went through Peter's attached document.
At the end of his document he lists some comments or concerns. I have copied them below with my own replies.
1. An error occurred during the submission of the questionnaire. The error message appeared just for a second and it seems that the session time expired. This was after a period of inactivity.
*This is a website error.*
2. In order to beat the short session time the we tried to enter in the information in parts by saving and modifying it. Unfortunately the “view modify” link is not working and an empty page comes up. The data cannot be modified; everything must be typed in again.
*This is a website error.*
3. If no data is entered the product gets a GOLD certification.
*This is a website error.*
4. Even with a detailed knowledge of OMII-UK software products the OMII employee carrying out the assessment found it difficult to answer some of the questions. The questions are open to various interpretations. Perhaps the answers should have been peer-reviewed (by OMII colleagues) in order to ensure correctness.
*Peer review may be a very good idea.*
6. Certified Open focuses mainly on the assessment of the “lock-in” factor.
*It's our sole focus, if I recall correctly. So that's OK.*
7. Certified Open is based on self-certifying principle, however the correctness of answers is not verified. Perhaps again peer review would enable a more objective assessment.
*I tend to agree. There is also the final authentication layer before the award of the formal certification.*
8. There are no references in the “Certified Open Product and Services Framework” document. The OMII staff member performing the assessment felt a short description of the evolution of the “Certified Open” idea would be beneficial for a better understanding of the assessment process.
*This paragraph was not entirely clear to me. However, I understand that the staff member wanted a contextualisation of the principle of Certified Open.*
9. We suggest that Certified Open should explain in more detail the motivation behind the “Certified Open” initiative by focusing on the benefits for software providers.
*Related to point 8, the staff appear to wish for an application of the principle of Certified Open to their organisation's perceived benefit from it's use. Like the previous point, this indicates that we are not yet selling the idea clearly enough. A marketing focus problem.*
10. The OMII employee carrying out the assessment suggested that Certified Open extend the questionnaire by additional categories for assessing the cost of software development and quantifying the number of potential customers.
*These are not lock-in issues. These are issues of development and deployment planning. (at least, that's my perspective)*
11. The evaluation of each component took a little longer than an hour. A significant benefit of the tool is that it encourages reflection on the aspects of the components that enable or damage openness.
*Excellent.*
12. The components that we chose were the most appropriate for this sort of evaluation, so our generally favorable assessment may be biased by that. Also, OMII has a mission to produce Open components. All our components are Open Source. This may also have biased our view.
*Perhaps we need to have methods of people stating potential bias? Full disclosure in a box preceding the primary form? This could later be used in any cases where charges of intentional distortion were levelled against a certified body.*