[IGFmaglist] suggested workshop grading scale descriptions - merging of proposals

Virginia Paque virginiap at diplomacy.edu
Wed Apr 22 10:01:22 EDT 2015


Thanks Susan, and everyone for this valuable thread. Particularly as a new
MAG member, I find the workshop evaluation process to be very important,
and a bit daunting.

Since we have a firm deadline for evaluations, some of us have begun
evaluating, using our own Excel / database spreadsheets.

Did I miss a link or instructions to an online evaluation format that we
will use for final evaluations? Can someone please summarise the process we
will use? I did attend the evaluation training webinar, but I must have
missed this very important practical point.

Thanks,
Ginger

Ginger (Virginia) Paque
DiploFoundation

*DiploFoundation upcoming online courses:* http://www.diplomacy.edu/courses

On Tue, Apr 21, 2015 at 2:53 AM, Susan Chalmers <susan at chalmers.associates>
wrote:

> Greetings all,
>
> Many thanks to all who have contributed to the various discussions on
> workshop feedback and mergers. In terms of the tasks presently before us:
>
> 1. grading workshop proposals,
> 2. highlighting them as possible mergers, and
> 3. providing feedback for workshops scored lower than three points....
>
> thank you to Flavio for providing the 7 standardized reasons for feedback.
>
> To clarify, last year MAG members were asked to provide personalized,
> written feedback for all workshop proposals they scored 3 or below. For
> those who did it, this was incredibly time-intensive. The 7 standardized
> reasons should reduce the time involved and provide ample information for
> the Secretariat to in turn provide to workshop proponents. The text box
> will remain available if we want to provide personalized comments.
>
> I also see general consensus in favour of Flavio's suggestion to add a
> "merger" tickbox on the evaluation sheet, that will not factor into the
> workshop score. It is for us to mark proposals if we believe that they are
> good merger candidates, and discuss why in person at the meeting.
>
> In terms of mergers, thank you to all for your general thoughts and
> concerns, sharing your past experiences, and to those who have shared
> possible solutions. In terms of the latter, Avri, I really liked your
> proposal to invite people who are interested in merging with others to
> self-organise right now, before grading, but given that discussions on the
> MAG take time to find consensus, I'm not sure if we have sufficient time
> before the grades are due.
>
> I propose that the Secretariat implement the technical changes necessary
> for the workshop evaluation sheets (i.e. adding the standardized reasons
> and the merger tickbox), and that, before the MAG meeting, we develop
> lightweight merger criteria, recognising that:
>
>    - if two groups are *asked* to merge, then either the Secretariat or a
>    2-MAG member team must assist the discussion Imbalances between groups, as
>    the Chair and others have pointed out, sometimes means that a guiding hand
>    is necessary;
>    - if two groups *want* to merge on their own accord, then they should
>    be able to signal their willingness and interest to the MAG before the
>    meeting; and,
>    - as Mark succinctly stated, mergers should occur based upon:
>       - 1) obvious similarity with other proposal(s) - or
>       - 2) having inherent merit but also weakness(es) that would be
>       addressed through merging with another similar but stronger proposal.
>
> Above all, because this is a discussion in process, the "merger"
> question should not figure into our workshop scoring. I think that's rather
> important to make clear.
>
> Sincerely,
> Susan
>
>
>
>
>
>
>
> Susan Chalmers
> susan at chalmers.associates
>
> *CHALMERS* & ASSOCIATES
> http://chalmers.associates
>
> On Wed, Apr 15, 2015 at 9:59 PM, Makane Faye <fmakane at gmail.com> wrote:
>
>> Dear Colleagues,
>>
>> The evaluation is not easy and will be further complicated if we want to
>> add comments on how to improve submissions on the low scorers. This will
>> also take a big amount of time.
>>
>> What we did last year was to provide a justification to low scorers on
>> why the project was not selected. We also had decided to provide mentoring
>> to submissioners who needed support and requested for it to have a better
>> chance for next IGF(s).
>>
>> On merger issues we need to take into account what Janis, Subhi and Mark
>> have said and come up with a mid way proposal. During the past 2 IGFs, most
>> of the recommendations for merger were not accepted by the workshop
>> organizers, hence we need to come with a more subtile way of requesting for
>> mergers independently from the scores of workshops.
>>
>> Best regards,
>>
>> ----------------
>>
>> Makane Faye (Mr.)
>> Chief, Knowledge Services Section  ---
>> United Nations Economic Commission for Africa (ECA) ---
>> Limpopo Building ---
>> P.O. Box 3001 Addis Ababa, Ethiopia ---
>> Tel: +251 11 5443563; Ext. 33563 // Fax: +251 11 5514416 / +251 11
>> 54411458 ---
>> Email: mfaye at uneca.org; faymakane at gmail.com ---
>> http://facebook.com/makane.faye.eca // twitter: @makaneeca // Skype:
>> makaneaddis
>>
>> Makane Faye (M.)
>> Chef de la Section des Connaissances ---
>> Commission Economique des Nations Unies pour l’Afrique (CEA) ---
>> Bâtiment Limpopo ---
>> BP. 3001 Addis Abeba, Ethiopie ---
>> Tel: +251 11 5443563; Ext. 33563 // Fax: +251 11 5514416 / +251 11
>> 54411458 ---
>> Courriel: mfaye at uneca.org; faymakane at gmail.com ---
>> http://facebook.com/makane.faye.eca // twitter: @makaneeca // Skype:
>> makaneaddis
>>
>>
>>
>>
>>
>>
>> On Wed, Apr 15, 2015 at 12:18 PM, Mark Carvell <
>> mark.carvell at culture.gov.uk> wrote:
>>
>>> Marilyn
>>>
>>> I think you're right: need to retain a flexible and pragmatic approach -
>>> while also not bending over backwards to be generous and keeping the MAG's
>>> and the Secretariat's workload under control!
>>>
>>> I said on the call yesterday that I would not find it easy to
>>> differentiate between the negative scores in the range of 1-3 in the list
>>> in Susan's e-mail. From my experience last year, there were obvious cases
>>> of failure to meet essential IGF criteria so rejection in those cases was
>>> clear (need to provide feedback so they learn for next year). I think
>>> "3.Borderline" and "4. could be accepted" seem to me also to be difficult
>>> to differentiate as scores - what exactly is the difference? I would prefer
>>> an approach along the lines of
>>> 1. Rejection due to failure by applicant to meet all essential criteria
>>> and nothing to be gained by trying to fix the proposal - ACTION:
>>> Secretariat provides feedback on reasons for rejection;
>>> 2. Merger candidate due to i. obvious similarity with other proposal(s)
>>> - or ii) having inherent merit but also weakness(es) that would be
>>> addressed through merging with another similar but stronger proposal -
>>> ACTION:  MAG-led dialogue with all applicants involved in mergers to help
>>> resolve;
>>> 3. Acceptable on topic/substance/potential contribution to IGF outputs
>>> but conditional on improvements - e.g. achieving geo-diversity , gender
>>> balance etc in a confirmed session panel or whatever....- ACTION:
>>> Secretariat-led dialogue with applicants to resolve, prior to stage 3 face
>>> to face discussion and decision by MAG on whether to proceed to confirm
>>> acceptance;
>>> 4. Accept as a robust proposal consistent with all IGF criteria.
>>>
>>> I think another flexibility issue is uniqueness if for example the
>>> proposal is about an emerging issue that is not covered by any other
>>> proposal, should we mark it down and reject it if it falls down on the
>>> other criteria; or should the MAG aim to work the applicants to upgrade it
>>> because of the potential value on "new" substance in particular I can't
>>> remember specific examples last year but I was very mindful of this in my
>>> evaluations: I would score uniqueness and its value for the IGF highly even
>>> if it was weak on other format and delivery criteria.
>>>
>>> I hope these comments and suggestions are helpful.
>>>
>>> best regards
>>>
>>> Mark
>>>
>>> Mark Carvell
>>> UK Government: Global Internet Governance Policy
>>>
>>> On 14 April 2015 at 15:02, Marilyn Cade <marilynscade at hotmail.com>
>>> wrote:
>>>
>>>>
>>>> This does not show the flexibility that the MAG should convey.  When
>>>> the Secretariat publishes this , in the MAG's name, you are saying 1, and
>>>> it has your name on it, as a MAG member.
>>>>
>>>> This is unfair and unkind to the submitter.
>>>>
>>>> Let's show a little compassion and a little innovation in the rating.
>>>>
>>>> 1) proposal does not meet [name the criteria] e.g. not linked to the
>>>> sub themes, lacks geo diversity
>>>> 2) proposal does not meet [name the criteria]
>>>> 2.5) Proposal has potential, should be merged with other proposals [
>>>> offer guidance on which category]
>>>> 3) Improvements in S, Y, A will enable acceptance, that may include
>>>> merging...
>>>> 4) Small changes needed - specify what those are
>>>> 5) Approved
>>>>
>>>>
>>>>  Dear MAG colleagues,
>>>>
>>>>  Based upon input from the last virtual meeting, how do the following
>>>> grading scale descriptions sit with everyone?
>>>>
>>>>  1 = proposal has serious problems
>>>> 2 = should not be accepted
>>>> (2.5 = should be merged with another)
>>>> 3 = borderline
>>>>  4 = could be accepted
>>>>  5 = must be accepted
>>>>
>>>>  Please don't hesitate to share your thoughts.
>>>>
>>>>  Sincerely,
>>>> Susan
>>>>
>>>>
>>>> Susan Chalmers
>>>> susan at chalmers.associates
>>>>
>>>> *CHALMERS* & ASSOCIATES
>>>> http://chalmers.associates
>>>>
>>>>
>>>> _______________________________________________
>>>> Igfmaglist mailing listIgfmaglist at intgovforum.orghttp://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>>>>
>>>>
>>>>
>>>> _______________________________________________ Igfmaglist mailing list
>>>> Igfmaglist at intgovforum.org
>>>> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>>>>
>>>> _______________________________________________
>>>> Igfmaglist mailing list
>>>> Igfmaglist at intgovforum.org
>>>> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>>>>
>>>>
>>>
>>>
>>> --
>>> Mark Carvell
>>> Global Internet Governance Policy
>>> Department for Culture, Media and Sport
>>> mark.carvell at culture.gov.uk
>>> tel +44 (0) 20 7211 6062
>>>
>>> _______________________________________________
>>> Igfmaglist mailing list
>>> Igfmaglist at intgovforum.org
>>> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>>>
>>>
>>
>> _______________________________________________
>> Igfmaglist mailing list
>> Igfmaglist at intgovforum.org
>> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>>
>>
>
> _______________________________________________
> Igfmaglist mailing list
> Igfmaglist at intgovforum.org
> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://intgovforum.org/pipermail/igfmaglist_intgovforum.org/attachments/20150422/52a7e2f8/attachment-0001.html>


More information about the Igfmaglist mailing list