[IGFmaglist] Evaluating workshop proposals - Standardized reasons for scores of 3 or less

Susan Chalmers susan at chalmers.associates
Fri Apr 17 00:38:38 EDT 2015


Dear Flávio, Mark,

Flávio, I appreciate your approach, thank you kindly for suggesting
standardised reasons for workshop proposals that have a score of 3 or less.

For those workshops which must be respectfully declined - which will be
about 150 - proponents will be provided with this feedback.

These tick box options will exist alongside a free text box, where we can
submit additional thoughts on the proposal. Please word your thoughts
constructively and compassionately for the proponent.

Unless there are any strenuous objections, I suggest that we submit these
to the Secretariat, so that the IT person can get started on adding these
to the website. Is that okay with everyone?

Mark, in regards to your observation, there are a few "positive
discrimination" elements - in favour of developing countries, first time
proposers, new session formats - upon which the MAG has found rough
consensus. MAG members are asked to reflect these elements in their overall
score. These elements are not individually weighted. That is a conversation
that the next MAG may wish to entertain.

Sincere regards,
Susan




On Wednesday, April 15, 2015, Mark Carvell <mark.carvell at culture.gov.uk>
wrote:

> Dear Flavio
>
> Many thanks. This list captures very effectively and comprehensively the
> key evaluation criteria so is very helpful and I support.
>
> I have just one query regarding 4 in both lists: I accept that "first-time
> proposers are preferred over repeat-proposers" serves to re-fresh active
> stakeholder participation in the IGF but I hope the intention is not that
> having previously submitted a proposal should be a decisive negative
> criterion when comparing with other similar proposals. If it were, we would
> have to explain that to the applicant.
>
> Kind regards
>
> Mark
>
> On 15 April 2015 at 11:33, Flavio Rech Wagner <flavio at inf.ufrgs.br
> <javascript:_e(%7B%7D,'cvml','flavio at inf.ufrgs.br');>> wrote:
>
>>  Dear MAG members
>>
>> As already mentioned by Susan, I volunteered during the yesterday's
>> virtual meeting on workshop evaluation to propose "standardized reasons" -
>> kind of "buttons" the evaluators could easily click on - for why a proposal
>> received a score of 3 or less.
>>
>> These reasons would match the "Considerations when Evaluating Proposals",
>> which have been already defined by the MAG and are copied at the end of
>> this message for your reference.
>>
>> Based on the "considerations", I suggest that the online evaluation form
>> includes two sets of "buttons". The reasons directly match each of the
>> considerations.
>>
>> For each proposal, each evaluator may click on one or more buttons, as
>> s/he thinks appropriate.
>>
>> * FIRST SET OF REASONS*
>>
>> The first set gives reasons that, after aggregation by the secretariat,
>> will be sent to the proposers to explain the final average scores their
>> proposals received (numbers before the reasons correspond to the numbers of
>> the "considerations").
>>
>> (1) The proposal is either (a) not well thought-through or (b) incomplete.
>> (2) The proposal is not relevant to Internet Governance.
>> (3) The proposal does not include either (a) a list of proposed speakers,
>> participating individuals and organizations, or (b) a description of how
>> different stakeholder perspectives will be represented.
>> (5) The workshop description is not consistent with the format listed.
>> (7) There is no diversity amongst the participants (gender, geography,
>> stakeholder group, perspective).
>> (9) The description does not clearly specify the Internet Governance
>> problem/question/challenge to be addressed during the workshop.
>> (10) The proposal does not include a well-considered plan for remote
>> participation.
>>
>> *SECOND SET OF REASONS*
>>
>> The second set corresponds to reasons that justify the subjective score
>> given by the evaluator, in comparison with other proposals, but do not
>> necessarily represent a weakness of the proposal, so that these reasons do
>> not need to be sent to the proposers:
>>
>> (4) This is not the first time this individual or organization has
>> submitted a workshop proposal to the IGF.
>> (6) This is not a proposal for a new format (Break-out Group Discussions,
>> Debates, Flash Sessions, Birds of a Feather, Roundtables and Other formats
>> are encouraged over the Panel format).
>> (8) There is no participation from developing countries.
>>
>> I am looking for your feedback.
>>
>> Best
>>
>> Flavio
>>
>> ----------
>>
>>
>>
>> *Considerations when Evaluating Proposals *from
>>
>> http://www.intgovforum.org/cms/workshop-proposals/mag-workshop-review-and-evaluation-process-for-igf-2015
>>
>>  1.    Is the proposal well thought-through and complete?
>> 2.    Is the proposal relevant to Internet Governance?
>> 3.    Does the proposal contain a list of proposed speakers,
>> participating individuals and organizations, or a description of how
>> different stakeholder perspectives will be represented?
>> 4.    Is this the first time this individual or organization has
>> submitted a workshop proposal to the IGF? (first-time proposers are
>> preferred over repeat-proposers),
>> 5.    Is the Workshop description consistent with the format listed (for
>> example, if the format is Debate, then does the proposal describe how the
>> debate will be set up, with timings, etc.,  indicated, are all sides of the
>> issues represented)?
>> 6.    Is the proposal for a new format? (Break-out Group Discussions,
>> Debates, Flash Sessions, Birds of a Feather, Roundtables and Other formats
>> are encouraged over the Panel format),
>> 7.    Is there diversity amongst the participants (gender, geography,
>> stakeholder group, perspective)? (as a general matter, greater diversity is
>> encouraged),
>> 8.    Is there developing country participation? (as a general matter,
>> developing country participation is encouraged),
>> 9.    Does the description clearly specify the Internet Governance
>> problem/question/challenged to be addressed during the workshop?
>> 10.    Does the proposal include a well-considered plan for remote
>> participation?
>>
>>
>> _______________________________________________
>> Igfmaglist mailing list
>> Igfmaglist at intgovforum.org
>> <javascript:_e(%7B%7D,'cvml','Igfmaglist at intgovforum.org');>
>> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>>
>>
>
>
> --
> Mark Carvell
> Global Internet Governance Policy
> Department for Culture, Media and Sport
> mark.carvell at culture.gov.uk
> <javascript:_e(%7B%7D,'cvml','mark.carvell at culture.gov.uk');>
> tel +44 (0) 20 7211 6062
>


-- 


Susan Chalmers
susan at chalmers.associates

*CHALMERS* & ASSOCIATES
http://chalmers.associates
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://intgovforum.org/pipermail/igfmaglist_intgovforum.org/attachments/20150417/2a003d40/attachment-0001.html>


More information about the Igfmaglist mailing list