[IGFmaglist] Weighting of criteria, please send your input by March 25th, 12.00 UTC

Fri Mar 15 10:31:02 EDT 2019

Dear all!
I do not know what algebra Sylvia used, but in the case that all marks in all the criteria are the same, the average mark should (must) be that same mark. If all marks are 5, the average is 5, if all marks are 4, the average is 4...REGARDLESS OF THE WEIGHT OF THE CRITERIA!The difference happens when one or more of the marks are different. In that case, the higher mark is in the higher weight, the average is higher. And of course that the decimals matter, and that's why I am against different weights.And that's why I asked Secretariat if they could run a test on the last year results, applying fictional 10%, 20%, 30%, 40% weight for either 50 top scorers or 50 randomly selected WS to compare what are the changes in their rankings.  
I attached an excel file with some examples of the groups of the same marks in different criteria. Notice the difference in average mark inside the same grouping even though the marks are "same", but distributed differently. Please feel free to play with the weight, just enter the new numbers taking care that the total /sum/ of all the criteria MUST BE 100, and see how the average changes.
I repeat what I previously mentioned in the chat during the meeting, I am not in favor of the different weight of the criteria.

The first problem is that it would be very difficult to agree among the MAG what is "less" valuable, and what is "more" valuable. In extreme case, for me, it is one criterion, and I should like to have 50% weight, and for somebody else, it is another criterion and (s)he would like to have 60% for that criterion.

However, I think that the idea is worth exploring and it would not be late to make the final decision at the next meeting.

Nebojša Regoje

   On Thursday, 14 March 2019, 01:16:16 CET, Sylvia Cadena <sylvia at apnic.net> wrote: 

Thanks Jutta for preparing the question to the MAG so clearly.

At this stage the idea is to first get the MAG to decide if we should apply weight or not. If yes, a separate discussion about which weight to apply will take place, so the weights on the google sheet are only a reference.

The draft evaluation form currently under discussion at the WG is available for your comments here. Please bring your comments to the WG or add them to the form.

https://docs.google.com/spreadsheets/d/1TtWPeqkGaiZTLCKDoCnuyweo2_EhfKgCYbl8lkij6I8/edit?usp=sharing

By applying weights among the different 6 criteria, the difference on the average score are very small. If we don’t use weights and calculate the average score, for a proposal that scores 5 in all 6 criteria, the score will be 5 as all scores will have the same weight (around 16,6%). If we applied the proposed weights the score will be 4.5833. As you can see the difference is really marginal. But the decimals will be important to have the shortlist.

Regards,

Sylvia

________________________________________________________________________

________________________________________________________________________

Sylvia Cadena | APNIC Foundation - Head of Programs | sylvia at apnic.net | http://www.apnic.foundation

ISIF Asia, WSIS Champion on International Cooperation 2018 & 2019 | http://www.isif.asia | FB ISIF.asia | @ISIF_Asia | G+ ISIFAsia | 

6 Cordelia Street, South Brisbane, QLD,  4101 Australia | PO Box 3646 | +10 GMT | skypeID: sylviacadena | Tel: +61 7 3858 3100 |  Fax: +61 7  3858 3199

* Love trees. Print only if necessary.

From: chenai chair <cchair at researchictafrica.net>
Date: Thursday, 14 March 2019 at 1:15 am
To: Jutta Croll <jcroll at digitale-chancen.de>
Cc: MAG-public <Igfmaglist at intgovforum.org>
Subject: Re: [IGFmaglist] Weighting of criteria, please send your input by March 25th, 12.00 UTC

Thanks Jutta. 

May we have the link to the excel document that Sylvia created gives a brief on each criteria?

On 13 Mar 2019, at 17:06, Jutta Croll <jcroll at digitale-chancen.de> wrote:

Dear MAG members, 

following our virtual meeting today we would like to invite you to comment via the list on the issue of weighting of criteria.

At the virtual meeting of the working group WSP on the workshop process it was discussed whether or not the scores for the criteria in the workshop evaluation should be weighted differently, according to their agreed importance to the overall process. Various options on how or whether to weight the criteria were discussed. Although there was consensus that certain criteria (f.e. Diversity and Relevance) might merit a higher weight than others (f.e. format), it was also cautioned that differentiating criteria by weight would need due consideration of the consequences this might have on the overall ranking of workshop proposals. 

For your own deliberations I’d like to explain the calculations in comparison of last year and 2019.

In 2018 we had 4 criteria with equal weight, that means each score went into the overall score with 25 %.

in 2019 we have 6 criteria, that means each score will go into the overall score with 16,666 %

The 6 criteria are

·        Policy Question(s)

·        Relevance

·        Format

·        Diversity

·        Content

·        Interaction

The working group WSP will be happy to receive your feedback on weighting either differentiating or equal for each criterion by Monday, 25th, 12.00 UTC. The working group will take all your input in due consideration and process them before the next virtual MAG meeting on March 27th.

With kind regards

Jutta 

**********************************
Jutta Croll M. A.
Stiftung Digitale Chancen
- Projektleitung Kinderschutz und Kinderrechte in der digitalen Welt - 
- Vorsitzende des Vorstands / Chairwoman of the board - 
Chausseestr. 15 
D-10115 Berlin
Germany

Skype: jcrollSDC

Tel.: ++ 49(0)30-43727730
Mobil: ++ 49(0)163-5493-773
E-Mail: jcroll at digitale-chancen.de
URL: www.digitale-chancen.de

English Version: 
http://www.digitale-chancen.de/index.cfm/lang.2
**********************************

_______________________________________________
Igfmaglist mailing list
Igfmaglist at intgovforum.org
To unsubscribe or manage your options please go to http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org

 _______________________________________________
Igfmaglist mailing list
Igfmaglist at intgovforum.org
To unsubscribe or manage your options please go to http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://intgovforum.org/pipermail/igfmaglist_intgovforum.org/attachments/20190315/6a9cd506/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Marks.xlsx
Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Size: 10728 bytes
Desc: not available
URL: <http://intgovforum.org/pipermail/igfmaglist_intgovforum.org/attachments/20190315/6a9cd506/attachment.xlsx>