[IGFmaglist] suggested workshop grading scale descriptions - merging of proposals

Susan Chalmers susan at chalmers.associates
Tue Apr 21 03:53:44 EDT 2015


Greetings all,

Many thanks to all who have contributed to the various discussions on
workshop feedback and mergers. In terms of the tasks presently before us:

1. grading workshop proposals,
2. highlighting them as possible mergers, and
3. providing feedback for workshops scored lower than three points....

thank you to Flavio for providing the 7 standardized reasons for feedback.

To clarify, last year MAG members were asked to provide personalized,
written feedback for all workshop proposals they scored 3 or below. For
those who did it, this was incredibly time-intensive. The 7 standardized
reasons should reduce the time involved and provide ample information for
the Secretariat to in turn provide to workshop proponents. The text box
will remain available if we want to provide personalized comments.

I also see general consensus in favour of Flavio's suggestion to add a
"merger" tickbox on the evaluation sheet, that will not factor into the
workshop score. It is for us to mark proposals if we believe that they are
good merger candidates, and discuss why in person at the meeting.

In terms of mergers, thank you to all for your general thoughts and
concerns, sharing your past experiences, and to those who have shared
possible solutions. In terms of the latter, Avri, I really liked your
proposal to invite people who are interested in merging with others to
self-organise right now, before grading, but given that discussions on the
MAG take time to find consensus, I'm not sure if we have sufficient time
before the grades are due.

I propose that the Secretariat implement the technical changes necessary
for the workshop evaluation sheets (i.e. adding the standardized reasons
and the merger tickbox), and that, before the MAG meeting, we develop
lightweight merger criteria, recognising that:

   - if two groups are *asked* to merge, then either the Secretariat or a
   2-MAG member team must assist the discussion Imbalances between groups, as
   the Chair and others have pointed out, sometimes means that a guiding hand
   is necessary;
   - if two groups *want* to merge on their own accord, then they should be
   able to signal their willingness and interest to the MAG before the
   meeting; and,
   - as Mark succinctly stated, mergers should occur based upon:
      - 1) obvious similarity with other proposal(s) - or
      - 2) having inherent merit but also weakness(es) that would be
      addressed through merging with another similar but stronger proposal.

Above all, because this is a discussion in process, the "merger"
question should not figure into our workshop scoring. I think that's rather
important to make clear.

Sincerely,
Susan







Susan Chalmers
susan at chalmers.associates

*CHALMERS* & ASSOCIATES
http://chalmers.associates

On Wed, Apr 15, 2015 at 9:59 PM, Makane Faye <fmakane at gmail.com> wrote:

> Dear Colleagues,
>
> The evaluation is not easy and will be further complicated if we want to
> add comments on how to improve submissions on the low scorers. This will
> also take a big amount of time.
>
> What we did last year was to provide a justification to low scorers on why
> the project was not selected. We also had decided to provide mentoring to
> submissioners who needed support and requested for it to have a better
> chance for next IGF(s).
>
> On merger issues we need to take into account what Janis, Subhi and Mark
> have said and come up with a mid way proposal. During the past 2 IGFs, most
> of the recommendations for merger were not accepted by the workshop
> organizers, hence we need to come with a more subtile way of requesting for
> mergers independently from the scores of workshops.
>
> Best regards,
>
> ----------------
>
> Makane Faye (Mr.)
> Chief, Knowledge Services Section  ---
> United Nations Economic Commission for Africa (ECA) ---
> Limpopo Building ---
> P.O. Box 3001 Addis Ababa, Ethiopia ---
> Tel: +251 11 5443563; Ext. 33563 // Fax: +251 11 5514416 / +251 11
> 54411458 ---
> Email: mfaye at uneca.org; faymakane at gmail.com ---
> http://facebook.com/makane.faye.eca // twitter: @makaneeca // Skype:
> makaneaddis
>
> Makane Faye (M.)
> Chef de la Section des Connaissances ---
> Commission Economique des Nations Unies pour l’Afrique (CEA) ---
> Bâtiment Limpopo ---
> BP. 3001 Addis Abeba, Ethiopie ---
> Tel: +251 11 5443563; Ext. 33563 // Fax: +251 11 5514416 / +251 11
> 54411458 ---
> Courriel: mfaye at uneca.org; faymakane at gmail.com ---
> http://facebook.com/makane.faye.eca // twitter: @makaneeca // Skype:
> makaneaddis
>
>
>
>
>
>
> On Wed, Apr 15, 2015 at 12:18 PM, Mark Carvell <
> mark.carvell at culture.gov.uk> wrote:
>
>> Marilyn
>>
>> I think you're right: need to retain a flexible and pragmatic approach -
>> while also not bending over backwards to be generous and keeping the MAG's
>> and the Secretariat's workload under control!
>>
>> I said on the call yesterday that I would not find it easy to
>> differentiate between the negative scores in the range of 1-3 in the list
>> in Susan's e-mail. From my experience last year, there were obvious cases
>> of failure to meet essential IGF criteria so rejection in those cases was
>> clear (need to provide feedback so they learn for next year). I think
>> "3.Borderline" and "4. could be accepted" seem to me also to be difficult
>> to differentiate as scores - what exactly is the difference? I would prefer
>> an approach along the lines of
>> 1. Rejection due to failure by applicant to meet all essential criteria
>> and nothing to be gained by trying to fix the proposal - ACTION:
>> Secretariat provides feedback on reasons for rejection;
>> 2. Merger candidate due to i. obvious similarity with other proposal(s) -
>> or ii) having inherent merit but also weakness(es) that would be addressed
>> through merging with another similar but stronger proposal - ACTION:
>>  MAG-led dialogue with all applicants involved in mergers to help resolve;
>> 3. Acceptable on topic/substance/potential contribution to IGF outputs
>> but conditional on improvements - e.g. achieving geo-diversity , gender
>> balance etc in a confirmed session panel or whatever....- ACTION:
>> Secretariat-led dialogue with applicants to resolve, prior to stage 3 face
>> to face discussion and decision by MAG on whether to proceed to confirm
>> acceptance;
>> 4. Accept as a robust proposal consistent with all IGF criteria.
>>
>> I think another flexibility issue is uniqueness if for example the
>> proposal is about an emerging issue that is not covered by any other
>> proposal, should we mark it down and reject it if it falls down on the
>> other criteria; or should the MAG aim to work the applicants to upgrade it
>> because of the potential value on "new" substance in particular I can't
>> remember specific examples last year but I was very mindful of this in my
>> evaluations: I would score uniqueness and its value for the IGF highly even
>> if it was weak on other format and delivery criteria.
>>
>> I hope these comments and suggestions are helpful.
>>
>> best regards
>>
>> Mark
>>
>> Mark Carvell
>> UK Government: Global Internet Governance Policy
>>
>> On 14 April 2015 at 15:02, Marilyn Cade <marilynscade at hotmail.com> wrote:
>>
>>>
>>> This does not show the flexibility that the MAG should convey.  When the
>>> Secretariat publishes this , in the MAG's name, you are saying 1, and it
>>> has your name on it, as a MAG member.
>>>
>>> This is unfair and unkind to the submitter.
>>>
>>> Let's show a little compassion and a little innovation in the rating.
>>>
>>> 1) proposal does not meet [name the criteria] e.g. not linked to the sub
>>> themes, lacks geo diversity
>>> 2) proposal does not meet [name the criteria]
>>> 2.5) Proposal has potential, should be merged with other proposals [
>>> offer guidance on which category]
>>> 3) Improvements in S, Y, A will enable acceptance, that may include
>>> merging...
>>> 4) Small changes needed - specify what those are
>>> 5) Approved
>>>
>>>
>>>  Dear MAG colleagues,
>>>
>>>  Based upon input from the last virtual meeting, how do the following
>>> grading scale descriptions sit with everyone?
>>>
>>>  1 = proposal has serious problems
>>> 2 = should not be accepted
>>> (2.5 = should be merged with another)
>>> 3 = borderline
>>>  4 = could be accepted
>>>  5 = must be accepted
>>>
>>>  Please don't hesitate to share your thoughts.
>>>
>>>  Sincerely,
>>> Susan
>>>
>>>
>>> Susan Chalmers
>>> susan at chalmers.associates
>>>
>>> *CHALMERS* & ASSOCIATES
>>> http://chalmers.associates
>>>
>>>
>>> _______________________________________________
>>> Igfmaglist mailing listIgfmaglist at intgovforum.orghttp://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>>>
>>>
>>>
>>> _______________________________________________ Igfmaglist mailing list
>>> Igfmaglist at intgovforum.org
>>> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>>>
>>> _______________________________________________
>>> Igfmaglist mailing list
>>> Igfmaglist at intgovforum.org
>>> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>>>
>>>
>>
>>
>> --
>> Mark Carvell
>> Global Internet Governance Policy
>> Department for Culture, Media and Sport
>> mark.carvell at culture.gov.uk
>> tel +44 (0) 20 7211 6062
>>
>> _______________________________________________
>> Igfmaglist mailing list
>> Igfmaglist at intgovforum.org
>> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>>
>>
>
> _______________________________________________
> Igfmaglist mailing list
> Igfmaglist at intgovforum.org
> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://intgovforum.org/pipermail/igfmaglist_intgovforum.org/attachments/20150421/ed3ba585/attachment-0001.html>


More information about the Igfmaglist mailing list