[IGFmaglist] Proposal for Modification for the Workshop Review and Evaluation Process

Lynn St.Amour Lynn at Internet-Matters.org
Tue May 31 10:42:28 EDT 2016


Thank you to Rasha, and all who contributed to this proposal.  

In order to advance the discussion for our MAG call, the Secretariat and I have tried to address a few of the questions here (where the responses are known)  

- Yes, proposals are identified as being submitted by a 1st time proposer (Rasha & Renata)
- the MAG does work towards an ideal number of workshops and this number will be known (Ginger)
- criteria for MAG members participation: emerging consensus seems not to change significantly from last year so little to no impact on WS proposals (Renata)

Open for discussion:

1 - will it be possible to put a proposal in a provisional status or have a second evaluation? (Ginger, Renata) 
2 - should workshops be evaluated in groups with similar themes (Ginger)
3 - do we need to assign weights to criteria to signal importance (Zeina)
4 - can MAG members review more than the assigned number of workshops? (Flávio)
5 - what feedback will be shared with proposers: text as last year, average ratings by criteria? (Rasha, Renata)

There are several new suggestions in this proposal and it may be easier to separate them out for discussion/approval:

1 - Increased focus/aggregation of criteria (note: all previous criteria are included)
2 - Scoring on each of the 4 (aggregated) criteria vs. one score encompassing all the criteria
3 - Each WS proposal will be reviewed by a subset of MAG members, proposal suggests 8 MAG members with 2 from each SH group, though this could be increased 

If you are not able to make the June 1st call, please send your comments to the list in order to help us assess consensus.  It is important that we hear from you.

Best regards,
Lynn

> On May 30, 2016, at 10:51 AM, Ginger Paque <VirginiaP at diplomacy.edu> wrote:
> 
> Thanks to those who worked on this, especially to Rasha for pushing it. I have to admit I was not sure we could find a way to implement a strategy of not having each MAG member evaluate each proposal, but I like this in general. I have still a few questions:
> 
> 1. Will there be a possibility to put a proposal in a provisional status, allowing for correcting some details, such as balance on the panel (gender/stakeholder/other, if appropriate and suggested); a clear proposal for including remote/online participants, and other necessary details?
> 
> 2. Will we work towards an ideal number of workshops? Should we be thinking about comparing workshops, if we know that only half of them can practically be accepted?
> 
> 3. Should workshops be evaluated in groups with similar themes, so that the 'best' proposals in certain thematic areas can be approved?
> 
> Best regards,
> Ginger
> 
> Virginia Paque
> DiploFoundation
> 
> Upcoming online courses: Humanitarian Diplomacy, 21st Century Diplomacy, Diplomatic Law: Privileges and Immunities, Internet Technology and Policy: Challenges and Solutions, Multilateral Diplomacy. http://www.diplomacy.edu/courses
> 
> 
> On Mon, May 30, 2016 at 7:46 AM, Renata Aquino Ribeiro <raquino at gmail.com> wrote:
> Dear Rasha and all
> 
> Thanks for submitting this proposal on workshop evaluation to MAG
> members analysis.
> 
> Proposal:
> http://intgovforum.org/pipermail/igfmaglist_intgovforum.org/attachments/20160529/77dcfc15/attachment-0003.docx
> 
> Along the main lines of the proposal, it provides a good way to modify
> the evaluation and streamline the process.
> I am in agreement.
> Thanks also to the team who participated on this.
> 
> I do have a few doubts, one of them to Secretariat
> 
> * ****Secretariat please see*****
> From the text:
> "The secretariat will provide information on whether or not this is a
> debut (first time) proposal. There could be a separate pool for debut
> presentations, or a certain number of points could be added to a debut
> presentation. Such point value would be determined for the first year
> once all the scores come in. This could be done at the NYC MAG
> meeting.
> Does the Secretariat have this info?
> 
> * Question to MAG Members (and all who would like to pitch in)
> From the text:
> "If all feedback is given to workshop proposers (including the
> scores), they would be able to know the strengths and weaknesses of
> their proposal just by looking at the different scores and knowing
> which items scored less than others. This would also help them make
> better proposals the following year."
> 
> Is it possible, to have a second chance evaluation for some workshops?
> Could this same feedback be directed to that?
> 
> My ask comes from the fact that sometimes works just need minor
> modifications to be adequate to presentation. Should this be taken
> into account here?
> 
> Also another issue
> As the MAG members are still discussing the adequate criteria for MAG
> members participation, it should be clear whether the rule applied
> would result in modification of workshop proposal and it would be
> important this is in the evaluation process publicly shared.
> 
> Thanks
> 
> Renata
> 
> 
> 
> 
> On Sun, May 29, 2016 at 7:23 AM, Dr. Rasha Abdulla <rasha at aucegypt.edu> wrote:
> > Dear MAG members,
> >
> > Following the Secretariat's green light, I have finalized the proposal for
> > modifying the Workshop Review and Evaluation Process. This proposal tackles
> > only the second stage of the review process, that of evaluation by MAG
> > members. The first stage (the Secretariat screening), as well as the third
> > stage (final decisions re borderline cases, mergers, etc) remain unchanged.
> >
> > I hope this proposal arrives at a middle ground for this year that takes
> > care of most of the concerns raised. It also reduces the subjectivity in
> > evaluation, and it considerably reduces the work load per MAG member. Many
> > thanks to Flavio, who suggested the work distribution among MAG members, and
> > to Susan for her comments on the whole process. I'm attaching the new
> > proposal on the second stage of reviewing as well as the current document
> > for the whole review process.
> >
> > In the interest of time before our next virtual meeting, and since there was
> > little interaction on the WG mailing list, I'm hereby offering the proposal
> > to the full list of MAG members for consideration. I request that the
> > Secretariat include this on Wednesday's meeting agenda if possible.
> >
> > Best regards.
> > Rasha
> >
> > Rasha A. Abdulla, Ph.D.
> > Associate Professor and Past Chair
> > Journalism and Mass Communication
> > The American University in Cairo
> > www.rashaabdulla.com
> > Twitter: @RashaAbdulla
> > <http://twitter.com/rashaabdulla>
> >
> >
> > _______________________________________________
> > Igfmaglist mailing list
> > Igfmaglist at intgovforum.org
> > http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
> >
> 
> _______________________________________________
> Igfmaglist mailing list
> Igfmaglist at intgovforum.org
> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
> 
> _______________________________________________
> Igfmaglist mailing list
> Igfmaglist at intgovforum.org
> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org





More information about the Igfmaglist mailing list