[IGFmaglist] Final Proposal for Session Evaluation Process

Renata Aquino Ribeiro raquino at gmail.com
Fri Mar 3 08:26:46 EST 2017


Hi

I'd also express concern about our timeline here.
The new process of workshop evaluation has been built by a working
group, publicized in the mailing list, received several inputs, some
were accepted, others were not (including mine).
It has been clearly communicated that evaluation improvements is an
ongoing process and that this year it would be closely observed, and
changes would be made, if necessary.
So do a test of a pilot to do an implementation and then official run?
If submission was in november, maybe it would be feasible, for 2018.


On Fri, Mar 3, 2017 at 1:46 PM, Lynn St.Amour <st.amour at bluewin.ch> wrote:
> Hi Mike, Laura, I don’t really understand how such a test would work.  We are all in different places/minds then 1 year ago - and relevance is so key - not just names, but more.
>
> Mike, what concretely do you propose?  One rating per session vs. 4 sub-ratings?  Or, the ability to do the sub-ratings and give a separate rating for the session as a whole (vs. an average).  That would be an interesting test - does the averaged rating match the “whole” rating?
>
> Lynn
>
>> On Mar 3, 2017, at 5:31 AM, Laura Watkins <laura.watkins at nominet.uk> wrote:
>>
>> Hi Mike,
>>
>> I agree that testing the proposed new structure is a good idea.  How about using a sample of last year’s submissions so we have a “real” pool of sessions to pick from and we’d be able to see if any of the selections would have been different?
>>
>> Thanks
>> Laura
>>
>>
>> Laura Watkins
>> Policy Executive
>>
>> <image001.png>
>>
>> Website | Twitter | Facebook
>>
>> DD: +44 (0)1865 332242    E: laura.watkins at nominet.uk
>>
>> Minerva House, Edmund Halley Road, Oxford Science Park, Oxford, OX4 4DQ, United Kingdom
>>
>>
>> Nominet UK. Registered in England and Wales No. 3203859
>>
>> This message is intended exclusively for the individual(s) to whom it is addressed and may contain information that is privileged, or confidential. If you are not the addressee, you must not read, use or disclose the contents of this e-mail. If you receive this e-mail in error, please advise us immediately and delete the e-mail. Nominet UK has taken every reasonable precaution to ensure that any attachment to this e-mail has been swept for viruses. However, Nominet cannot accept liability for any damage sustained as a result of software viruses and would advise that you carry out your own virus checks before opening any attachment.
>>
>>
>>
>> From: Igfmaglist [mailto:igfmaglist-bounces at intgovforum.org] On Behalf Of Michael R. Nelson
>> Sent: 03 March 2017 10:53
>> To: Dr. Rasha Abdulla <rasha at aucegypt.edu>; IGF Maglist <igfmaglist at intgovforum.org>
>> Subject: Re: [IGFmaglist] Final Proposal for Session Evaluation Process
>>
>> A big thank you to Rasha and everyone who contributed to the proposal for a new ranking process.  I'm impressed with the creative ideas for fixing many of the old problems with the proposal evaluation process (e.g. one panelist in five sessions, many similar proposals, etc.)
>>
>>
>> But, as I shared with other virtual participants yesterday, I am PASSIONATELY opposed to forcing all the MAG members to weight the many criteria in the RFP in exactly the same way.  This will lead to very skewed and unfortunate consequences--especially because you are only going to consider moving up 5-10 proposals that were "just below" the cut-off line.  This would mean the most controversial proposals would never be considered.  Likewise, panels with mediocre rankings that could ONLY be done in Geneva because the proposed panelists would be very unlikely to travel to a remote site.  And at the end of the day, it will be very hard to explain the weightings used in the ranking process.
>>
>>
>>
>> To test the proposed ranking mechanism, I would suggest we do an "alpha test" by considering a handful of hypothetical panel proposals and see how the system actually works.  Here are two:
>>
>>
>> Example 1:  “Who Makes Money in Cyberspace”
>>
>> Panelists (all confirmed):
>>
>> Bill Gates (USA), founder, Microsoft
>> Eric Schmidt (USA), Chairman, Alphabet and Google
>> Rex Tillerson (USA), US Secretary of State
>> President Xi (CHINA)
>>
>> Moderator:  John Markoff (USA), legendary technology reporter (NY Times)
>>
>> MY RANKING:  I suspect the ranking would be quite poor.  Old topic that really isn't central to Internet Governance.  Only two stakeholder groups.  TERRIBLE gender and geographic diversity
>>
>>
>>
>> Example 2:  "How Can the United Nations Use the Internet Better?
>>
>>
>> Moderator:
>> Klaus Schwab (SWITZERLAND, in Geneva)  President, World Economic Forum
>> Panelists (all confirmed):
>>
>> Filippo Grandi (ITALY but in Geneva) United Nations High Commissioner for Refugees
>> Zeid Ra’ad Al Hussein (ALGERIA but in Geneva), United Nations High Commissioner for Human Rights
>>
>> Francis Gurry (AUSTRALIA but in Geneva), Director General, World Intellectual Property Organization (WIPO)
>> MY RANKING:  I think the ranking would be well below the "cut-off line".  The topic, while important, is about using the Internet not "governing" it.  There is NO geographic diversity and minimal stakeholder diversity.
>>
>> How would others rank these proposals?  Any other hypothetical proposals we could use to test the ranking system?
>>
>> MICHAEL R. NELSON
>> Public Policy
>> Cloudflare
>> Washington, DC
>> 703-598-5187 cell
>> Twitter: @MikeNelson
>>
>>
>>
>> On Thursday, March 2, 2017 7:07 PM, Dr. Rasha Abdulla <rasha at aucegypt.edu> wrote:
>>
>>
>> Dear colleagues,
>>
>> Attached please find the final version of the proposal on the session evaluation process.
>>
>> Thank you so much for your support, and for the constructive discussion that was carried on during our meetings today. Your feedback was incorporated within this final version, and only serves to make it better.
>>
>> Once again, I want to thank the members of the working group for their diligent efforts.
>>
>> All the best.
>> Rasha
>>
>> Rasha A. Abdulla, Ph.D.
>> Associate Professor and Graduate Director
>> Journalism and Mass Communication
>> The American University in Cairo
>> www.rashaabdulla.com
>> Twitter: @RashaAbdulla
>> <http://twitter.com/rashaabdulla>
>>
>> _______________________________________________
>> Igfmaglist mailing list
>> Igfmaglist at intgovforum.org
>> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>>
>>
>> _______________________________________________
>> Igfmaglist mailing list
>> Igfmaglist at intgovforum.org
>> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>
>
> _______________________________________________
> Igfmaglist mailing list
> Igfmaglist at intgovforum.org
> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org




More information about the Igfmaglist mailing list