>> GUY BERGER:  Good morning to everybody!  Thank you for coming to this important session. 

I'm Guy Berger, and I work for UNESCO for Freedom of Expression and Media Development.

We have interesting people on my panel, and we have my codirector sitting at the side over here that will make final remarks at the end.  I will make introductory remarks and hand it over to the different people.

You hear about the concept of Internet universality, some of you may have heard about it at other places, times, IGFs, and now it is taking a step forward with the indicators that may make this helpful.  What's interesting about this adoption from 2015, in the universality, it is based on four principles, and why we thought ROAM was important was because rights, it is very important.  Rights without openness on the Internet is no Internet.  Rights without accessibility, it is a bit empty, and the whole package really works best when you have multistakeholder participation, key decisions that are shaping the Internet and Internet policies.

That's why you have this holistic perspective, ROAM.  What happens, we have a mandate from the Member States to try to take this concept, it is helpful in saying how we can think about the Internet as the whole elephant rather than feeling a leg here, an arm ‑‑ elephants, they don't have arms, a trunk, a tail, this gives you the per expectative of the elephant as a whole.  The value of getting this concept to actually have the ability to take shape on the ground, it is that UNESCO wanted to say how can we make a useful tool for voluntary application by countries to take an assessment of the Internet in his holistic character and see where improvements can be made.

In the elephant, of course, you want the elephant to walk on all four legs, not to have one leg like accessibility dragging behind, the elephant stumbling and the elephants should be balanced on all four legs but the elephant, it has a stomach and we're thankful at UNESCO to see that and to the Internet Society, we have an Internet Society that's a big support.  You have helped the elephant get the nutrition to move forward a bit.

I want to acknowledge, we have an advisory board of International experts, we have Andrea, anyone else here that's on the international advisory board?  Stephen, thank you for the feedback.

We have the group that we had contracted and they have been doing with us consultations worldwide and we have 25 consultations so far in different face‑to‑face and we have a website in six languages and we have some submissions.  I met a young man here, he said he made a submission.  I said yes, it is taken into account!  Fantastic.

That was phase I, it was what was important to mer.  We're looking at rights, not measuring every single right.  Each one could be a PhD dissertation with just one part of an indicator.  We asked people what right should we look at.  We got various suggestions and when we try to find out what people have in common, what's the motivation, logic, what's practical, useful.  That's phase I, what should look at.

We're at phase II it where we populated this indicator template with some proposed indicators and on behalf of the APC, David on my left, he's the one that's done this population, so this is phase II now.  Phase I, what goes in, phase II, draft contents.

We will have some continued phases, this is after we get the comments, the indicators will be further improved, and then we want to pretest them and we want to pilot them and after the pretesting, improve them a bit more, after piloting, improve them a bit more.  By November next year we want to submit this to the UNESCO Member States saying these are road tested and we have consulted around the world, we have lots of wisdom, we have been in cities all over, we have been all over asking people what they think could be in the indicators.  We'll say to the Member States here, will you give this your blessing as a useful, voluntary tool with a UNESCO imprint on it that can be used in a multistakeholder fashion to make an assessment of the country and identify where improvements could be made.

With that introductory comment, let me introduce you to David Souter.  He can then jump straight into the substance of the issue.

David Souter is I think a very well‑known person to all of you.  He's a longstanding consultant to UNTAG, to ITU, UNESCO, a professor at LSE.

Please, introduce the subject and we'll get responses from the panel and we'll open it up to everybody.

>> DAVID SOUTER: Thank you very much.

What I'll try to do in about 15 minutes is give an introduction to a little bit more about the process of the project and introduction to the proposed framework and say a bit about the Consultation that's going to be ‑‑ that's already started for the second phase of the work.

I'm hoping to take no more than 15 minutes on this.

I'll begin with the next slide.

The root of the project, there were two roots of the project that's worth bearing in mind.  Guy talked about one of these, the UNESCO Internet universality approach, it was adopted I think this 2014 ‑‑

>> GUY BERGER: 15.

>> DAVID SOUTER: And it was built by 4 principles of ROAM and it is the second root as well, the of media development indicators that were adopted at another date, about ten years ago I think.  When you look at this particular framework here, you may say to yourself, there is a lot of indicators in this.  This is pretty much similar in kind of overall numbers and proposed indicators and style of content to the media development indicators that have been used successfully in around 30   countries over that time which I use myself in an analyses.  There is qualitative, quantitative, institutional, and they enable investigators to put together a Collage of evidence and they have a proven track record following a similar pattern.  The purpose of the indicators on the next slide, it is to support governments and other stakeholders and I think in three ways, first in developing a clear understanding of the national Internet environment, second, seeing where that meets the ROAN principles and how to address them and then to develop a policy development recommendation of proposals responding to the deficiencies.  It is about identifying constructive ways forward for individual countries.

It is not ‑‑ to be specific, it is not intended as a cross‑country comparison.  It’s not that kind of indicator framework.  There is a special focus on gender and on children and on young people and it is also developed within the framework of the Sustainable Development Agenda.

Guy has discussed the project timeline, it is on this slide.  The key point, there are two faces.  Phase I we had so far, extensive Consultation, 165 online contributions, 25 face‑to‑face fora, the second phase is beginning now and is on this particular set of draft proposals which have been developed from UNESCO research and the influences made in the first Consultation process.  Illustrated here, the website for that Consultation process.

There's two phases, two types of Consultation:  Online Consultation, face‑to‑face meetings such as this, plus a multistakeholder advisory Board of Experts which Guy also mentioned.

Let me turn to the ROAM framework.  There are four principles within the ROAM framework and five categories of indicators within the indicator framework that's been proposed here.  The ROAM framework is rights based, openness, accessibility to all indicators and multistakeholder indicators.  To those we have added a fifth group, cross‑cutting indicators.  The cross‑cutting indicators are meant to be very broad, public policy goals of the international community.  Gender equality, Sustainable Development, those principles cut across all of the principles.  Within that category, the issues raised within the category are a supplement to the overall indicator framework.

For example, in looking at the impact of the Internet environment from a gender perspective, that requires an assessment of all five categories, not to just those supplementary issues within the gender component of the cross‑cutting category.

The structure to the framework, I'll explain that now.

The structure is built around the five categories.  Within each category, there are a number of themes.  What's illustrated here are those themes within the rights category.  There are six themes within the rights category, the overall legal and regulatory framework, Freedom of Expression, Right to information, the Right of association and the Right of association and participation, privacy and economic social and cultural Rights.  So there are six themes within that category and within each of those themes, there are a number of questions which are asked and indicators associated with the questions.  One indicator or a number of indicators associated with the questions.

In the final framework, the documents, there is a section on sources and means of verification which will give guidance to potential users of the indicator framework on the places that they should look for, both the broad formulation of what they're trying to assess and also the specific information that they need to do an assessment.  One or two other points around this, the indicator framework includes three types of indicator, quantitative indicators.  The quantitative indicators are available to some areas of this, particularly accessibility to all but for many of the issues that are raised within the ROAM framework it is difficult to achieve quantitative indicators.  In any event, the quantitative evidence is limited in many countries.

There is extensive use of qualitative indicators where we're proposing that those undertaking an assessment use credible sources within the country concerned and develop their understanding of how far a particular goal or question is, issues in a particular question are achieved.

Finally, there are institutional arrangements indicators.  These are to do with whether or not there is a particular policy framework in place within a government, whether there is a particular incident within government and associated with that, how that policy or institution is implemented.  So there are three kinds of indicators.  I would say that the structure along themes, questions ‑‑ go to the next slide ‑‑ the themes, questions, indicators and there are sources and means of verification.

I have three more general points to make before I introduce the framework itself.  On the left‑hand side of this slide here, you will see a list of six criteria for selection of indicators.  They're criteria that we have adapted from those in the media development indicators and applied in this particular context.  It is very important to have consistent, coherent indicators throughout an indicator framework like this and for those to apply in a way that's feasible for researchers to undertake.  You will see the points that are raised here.  Each indicator addresses one issue, not multiple issues, measurable data should be reliable and allow comfortable decision‑making, quantitative where possible, desegregating information and possible to gather data at a reasonable cost in most countries.  That last one, it is particularly important.  We are trying to develop here something that can be used by researchers in very wide-ranging contexts but remain consistent across the contents and for that purpose we have to be aware that there are limits on what's included in the framework as well as scope and scale.  Indicator frameworks need to be sufficiently broad to allow this kind of collage of evidence to be gathered but not so broad that it is impossible for them, for the evidence to be gathered reasonably by sufficient ‑‑ by people within a mission with the limited resources that are available to them.  That does limit the number of indicators you can have.  The number in this current set of proposals, it is around the same in the media development indicators.  My feeling from experience, it is that that is pretty large, and that ‑‑ so that the temptation, it is always to add more.

Actually in terms of managing a process of using something like this, it is generally better to reduce rather than increase the numbers that you have.  You don't have to be comprehensive.

The other point that I make here, it is ‑‑ I have a couple of examples here of how the questions in indicators work.

The first on the left‑hand side, it is a question for Freedom of Expression section, so one question is Freedom of Expression guaranteed in law, respected in practice and widely exercised and three proposed indicators addressing that question.

On right hand side, a more quantitative question from the accessibility to all section of the proposal of broadband networks geographically available throughout the country and three, possible indicators addressing that, all of which that are in practice gathered by the ITU.

>> GUY BERGER: If I can jump in, are there another copies of the document?  If you would loo I can a copy, it is useful as David proceeds.

Sorry to interrupt you.  I think this is helpful.  Thank you so much to Stephen, he's a speaker and we asked him to do leg work for us.  Thank you.

David, I think let's jump back in and try to continue.

>> DAVID SOUTER: As you get the documents to look at, I would say this particular document, it has been developed through desk research and very extensive Consultation process both online and offline and it proposes questions and indicators that we feel and UNESCO feels meets the requirement here.  They are proposals, they're draft for discussions and for further development through the next Consultation process.  We're asking you in this second Consultation phase, that includes this meeting, to think about three things, the three questions on the screen there.

Are there additional themes, questions, indicators which you think should be included in the framework?  Bearing in mind, obviously, what I just said about the need to keep within the limits of what's feasible for researchers and the majority of countries.

Second, suggestions that you would make with the proposed themes that are in here.

Thirdly, what sources and means of verification would you recommend from your experience in relation to the questions and indicators that have been proposed?

We have our own ‑‑ doing our own work on sources and means of verification.  Advice from those that have experience in all different context will be immensely helpful in putting that together.

Now, obviously in this meeting here, we can't go through these on an individual indicator basis.  What we're hoping you will do, take this away, you will make your contributions online and here you will also make initial contributions to us on individual points or more general issues that we can take forward.

To say, this is a draft for Consultation, it is not meant to be a final document.  Everything you say will be taken down in writing and paid attention to over the next remaining part of the work.

What I'll now do, quickly run through the different sections of the proposed framework, that's the five categories plus one more I'll mention in a second and the themes within categories and examples of the questions that are being asked in each of them.

As a reminder, those are the five categories of the proposed framework.  To those five categories we have added one further set of indicators, those are contextual indicators.  Context matters when you're assessing a national Internet environment because it affects what's possible to do.  It affects how uses of the framework will interpret the ROAMX indicators and how they'll develop recommendations arising from them.  They're important factors in how the Internet has evolved within a country, and they're important factors in constraining the Policy Options which are available to governments and other stakeholders arising from that context.  It is crucial to understand that we're proposing 20 contextual characters, and there are 6 in the categories and all are derived from data beds produced by or gathered by U.N. agencies or other international organizations.  This is not something that's a burdensome requirement on those undertaking the assessments here.  It is a quick way of setting out where a country stands in relation to the rest of the world.

So as you turn to the categories themselves, the first category is rights.  In rights we have identified six themes.  The overall policy legal and regulatory framework and we have adopted that as the first theme in each of the four ROAM categories.  Five others, so Freedom of Expression, the Right to information, both of derived from Article 19 on Civil and Political Rights, the freedom of association of right to participate in public life which also derived from that covenant, privacy, likewise derived from that covenant, social, economic, cultural rights derived from the international covenant on economic, social and cultural rights.  I don't know why I changed the order of those words.

This is examples, one of each of the six themes of the issues which are addressed by the questions and indicators.  They're just examples to have a quick look at and they're illustrated.  This won't give you the overall scope to get the overall scope.  You need to look in the booklet.

Second category, openness.  Guy said a little bit about openness.  We have at the beginning of this, the policy, legal, the frameworks and the four subsidiary areas of open standards, open markets and competition and open content and open data.  Again, here are some examples, one from each of the five themes within the openness category of the issues that are covered within the proposed framework.

Third category, accessibility to all.  Again, beginning with the policy, legal and regulatory framework and taking five dimensions of accessibility so connectivity, usage, affordability, equitable access by which we mean the relationship between different sections of the community, sections of society, local content and language, competences and capabilities.

Again, here are some examples, one from each of the six themes.

The fourth category is multistakeholder participation where there is policy, legal, regulatory framework, national Internet Governance and international Internet Governance and some examples of those on this slide.

Finally, we have the cross‑cutting group, the cross‑cutting category in which there are five themes.  I said that this initially was intended to contain very broad, global policy goals, gender equity, the requirement of children, young people, Sustainable Development and to those, there are two other cross‑cutting themes, trust and security, legal and ethical aspects of the Internet.  The fifth category is meant to consist of things that cut across the other four, and these are important things that needs to be engaged with when thinking about the other four themes.  We hope they'll be investigated in depth in other words, and it will be important to not just look at the indicators here that are supplementary but at the indicators across the theme.  We mentioned specifically gender equity being particularly true of children, young people and Sustainable Development, that you can't get a picture of those themes just from these indicators, and you'll only get a picture of the themes from these indicators plus those on the other four themes.

Some examples of those that are here and those categories, one of the things I did mention earlier, I think it is worth mentioning, yes, it would be good for comprehensive assessments to be made across the ROAMX framework.  That's the primary intention of these.  We don't exclude in any sense the possibility of people undertaking narrower investigations looking at only particular categories or dimensions.  This is not meant to be fixed you do this or nothing, it is meant to be something that could be adapted and developed and used in different ways by different people to suit the different contexts in which they're working and their own different requirements.  That diversity is an important part of it.  It is not, after all intended to compare country A with country B with country C but intended to investigate each individual country and explore where it meets the ROAM principles, where it doesn't, what may need to be done to address it and to make the policy recommendations as a result.

We turn to the three questions that we're asking in the second Consultation, which are those there.  There are six language websites but at the moment it is only online in English but in the near future it will be available in six languages, six U.N. languages, and I'll hand the floor back over to you.

>> GUY BERGER: Thank you.

I see people that just came in, anyone else that needs a copy, please ‑‑ yeah, it is the same.  It is the same.  Okay.

We have asked a lot of people, but we need a short input from them and we'll throw it open and get back to them as well.

The first person we asked to think about so far the state of the project, particularly from a gender point of view, we have Anriette Esterhuysen who I mentioned before.  She has been Executive Director of APC for many years and now she's APC's director for policy and strategy and among many achievements, she has a BA in musicology.

Make us some music!  A short song!

>> ANRIETTE ESTERHUYSEN: Thank you very much.

I won't say too much because David gave a fairly detailed presentation.  I won't say much about ‑‑ more about gender.  What we're trying to do with this firstly, this is important, you will see often that there is reference to desegregated data.  Almost in every area you will find that we're looking for data that's desegregated by age, by gender, and in other forms as well.  I think part of this is us wanting to form a relationship with people who gather data in the country, the national statistical agencies, research, academic institutions, even people in the Internet national registries for example.  We're trying to encourage the practice of always of gathering data around the Internet and making sure that it is desegregated.

Dealing with gender across a cross cut, there is a set of indicators, themes around vendor, but you also find if you go he for example to accessibility, you will find that there is a theme ‑‑

>> What page?

>> ANRIETTE ESTERHUYSEN: If you go to page 17, which is Category A, accessibility to all, A being one in principle, and if you page to 19, you will see this theme D, and the accessibility, equitable access.  If you turn the page, you see the first area there, D2 it, is there a gender digital device in Internet access and use.

As David said, even if you go in, a country, a group of organizations, public sector, government, they wanted to look at just accessibility, they still have to look at the gender dimension and the equitable access from the gender perspective.

The crosscuts, they don't standalone.  They pour out in certain areas.  We also try to mainstream important areas such as gender, such as children and youth.

I think similarly with rights, there is a cross‑cutting area, right at the end of the book, page 29.

Trust and security.

The final crosscut group, E, the legal and ethical aspects of the Internet.  You will find that some of the very specific rights‑related issues, they're also covered there.

I think it that's really all there is to say.  I think we have tried to design this in such a way that it can work with slices, but even those slices, if you hone on just accessibility, it will still give you a fairly holistic picture, and that was a goal.

>> GUY BERGER: That was that was a sweet sum, short, to the point, informative.

The next commentator, it is Xue Hong from China Normal University.  The professor serves not only there but also in Arena Law School, the Microsoft Information Society and on the ICANN Council.

Thank you so much.  You'll give us according to my notes a few comments on the openness indicators in particular.  3 minutes please.  We'll move on and come back.

>> XUE HONG: Sure.

Quick comments on openness:  I'm particularly interested in open data and the open market.  Let's go to open data first on page 15.  It was stated, it the open data mentioned here, it refers to the publicly available data gathered by government.  I suggest it would be better to mention this is not about personal data, normally subject to privacy protection.

Turn the page to page 16.  I draw your attention that question E3 and E5, I think these two questions are related.  For E3, it is primarily about the easy access to these publicly available data sets and E5, it is about the people's use and share of the data.  I think these are two issues that should be connected not only enabling people to access the data easily but also to use and share the data, the public data.

E3, it mentioned including machine readable access, definitely, this is access data online.  It must be machine readable.  I suggest we add something like to provide user friendly data format to make it interoperable so it will enable the easy access.

I hope this is the meaning of the question, E3, 5.

For E4, it is extremely important indicator, the location and duration of the data rotation.  I have two questions actually.  One is about international standard.  I would be very happy to know what is the sources of the international standard that's available at all.  Also I suggested a few words on the requirements, restricting cross‑border data.

That's my quick comments.

>> GUY BERGER: That you are for the high‑quality comments and we'll ask David at the end if he has a few comments.  We'll take notes and certainly these are good points.

The next brief points are from Stephen Weber from IFLA.  We all know that International Federation of Library Associations and Institutions according to my notes.  Not only libraries, but they have 1400 members in over 100 countries, and Stephen is a supporter of information for all and a participant in many UNESCO activities.

We were going to ask you to speak specifically about the accessibility to all.

>> STEPHEN WYBER: Thank you very much.  Thank you for the opportunity to speak. 

Clearly libraries have formed an information network, and we're glad to bring the reflection of how this functions with the work of UNESCO.  As you saw with the complement, I'm sure you will hear a lot about the work taking place that this is shaping up to be an amazing resource, not only for just organizations like ours, but also for our 1400 members at the national level and their members and the 2 million institutions and many millions more professionals who are trying to think broadly about the work they're doing.

I think talking of the indicators gives an unfairly idea of the value of the work that it will provide.  Information that you'll connect, providing an analysis here, it is useful for everyone and especially if you focus on the confirmation of the verification of whether there are plans, but are plans being implemented, budgets available, is there a serious commitment, that will be utterly essential.

I won't do it on small points, I'll cover that in response to the Consultation.

I think that the access section provides a lot of material that we need to be able to understand what share of the population is suffering from information poverty at the moment.  We have talked a lot about people being underconnected.  For us we talk about information poverty, it is a part of an observation that's not able to access the information and the services required to fulfill the objectives to live a full, happy life and to take advantage of the possibilities for personal development.  Of course, I owe it to myself to talk about public access.  I caution to presenting to those that are unconnected, can't afford it, people use different access, public, home, mobile for different reasons, even home access expands, it the use of library facilities, it is not falling off.  It is important to avoid giving the impression that public access is a sort of poor man's Internet.  We shouldn't stigmatize people who need to use the Internet for free.

I think the reference to culture, this is in right section, important, talking about making access meaningful, that cultural element is really, really important.  I hope we'll dig deeper than talking about reference to policy on cultural heritage online.  We facilitate the preservation access and creation in a digital age, a lot of work is going on in UNESCO on this subject and I hope we'll draw on it.

Clearly, also we should talk about research, innovation and this is featured in Article 27, access to innovation, access to science, new ideas, we should be covering the board on this.  I also have to talk about IP.  We talk about enforcement, not policies.  This is interesting work presented at WIPO recently showing whereas we have been good at creating new rights in the digital age, we're not really keeping up in terms of the Exceptions and Limitation allowing people to use the materials.  Making had sure we're digging down, there is a lot of talk about this elsewhere.  I argue that copyright deserves more attention at IGF in general and had this is worth looking at.

On isn't for People with Disabilities, I hope we'll see evidence of what other actors are doing to make websites available.  I don't think I have ever gone on a government website for fun or for personal fulfillment.  Normally it is because I fill out my tax or something.  There is a broad range of websites that is really accessible for People with Disabilities and it is very important.

When talking about local content, we should be focusing on services.  It is not just accessing materials, it is accessing apps, accessing services, accessing an offer that means that the Internet is relevant, that people can do things rather than just consume online.  This will link in clearly with a focus on skills and actually the ability to create things online, which is super valuable making sure that it is not just ever‑consuming Western material.

So overall, I think that a lot of the elements are there.  It will be the interplay of the different indicators.  The interplay of the different information that's really going to tell a fantastic, interesting story about is access meaningful.

I know people are achieving their own objects for achieving health, personal development online.  This is really a great project.  We're really glad to be involved.

Thank you.

>> GUY BERGER: I hope you'll send your notes and give us even more detailed comments.  Excellent.

The next commentator, Dorothy Gordan, a former board member of greater comments and on the advisory council.  Thank you for coming.  She's deputy Chair of UNESCO's Information for All program. 

Dorothy, your 3 minutes on Access also, accessibility.

>> DOROTHY GORDAN: Let me say, I think the design is very impressive.  It allows us to manage huge quantity of information in a very logical way.  I was very pleased to look at it. 

I'm not going to restrict myself just to access, for example right to information, if we look at page 9, we have D2, which has a title can Civil Society organizations organize effectively online?  I found that a little bit too open‑ended.  You know, what's affective organization online?  Interestingly, on page 11, you've got data encryption and online anonymity protected in law and in practice I corrected that.  In a way, that's consistent with international rights agreements.  This is a rapidly evolving space.  Perhaps we'll have to have a more in-depth look at it.  It is changing so fast and we have had some very good discussions here on this.

Only page 13, does the government facilitate free and Open Source software?  I feel the word facilitates, it is a bit too open‑ended, but I love the indicators.  The indicators will allow us to really hone in.  And as you know, I'm a member of the Free and Open Source Foundation for Africa ‑‑ a plug if you're an African and not a member, please join.  Okay.

I think organizationally it will help us.  As I was going through, being able to track when I was in Open using the B1, B2, et cetera, maybe if you start with OB1, then it will always be clear.  This is the B1 referring to the Open, you know, something like that, just to help us.

I was very excited about theme D, open content.  D1, does the government actively promote access to knowledge through its policies for education, culture and science?  A tough one to mention in my view.

Would you say that a government is using taxpayers’ money to buy software that locks them in to a particular vendor, would that be promoting access to knowledge?  You see what I mean.  It is a can of worms in there.  Maybe we need to decide which worms we're chasing.

Under accessibility, we have A2, is there a legal right to access the Internet and online services?  That's page 17.

Above that, I see effective arrangements in place to monitor access and use of Internet.  You refer to the regular household surveys.  It any alternative to that using a technical tool that would allow us to do that rather than the household surveys in terms of use of the Internet?  I think you have alternative options because the household surveys in many places, it is very ‑‑ it opens up the question actually of who ‑‑ the data sources we'll be looking at, but also which particular sets of organizations, individuals will actually be responsible for giving you the information needed for the indicators.  On the one I mentioned first, A2, the legal right to access the Internet and online services, tricky.  Tricky.  What do you do if permission has been given for 0 rating of different kinds?  It is tricky.

>> GUY BERGER: If I can stop you there for a moment.

>> XUE HONG: Can I just finish?

>> GUY BERGER: One more challenge.

>> XUE HONG: The big challenge we have is on the capabilities and competencies (Dorothy Gordan) under F3, the particular skills by skill settlement desegregated the proportion of the workforce using ICTs in the workplace.  These issues are challenges for everybody.  We haven't got clear definitions.  I can just see it being impossible for you to really get the information.

Let me say, I really admire the amount of work that's gone in here and it is going to be delightful to take it to the country level and to start the debate going.

Thank you.

>> GUY BERGER: Thank you.  That's really wonderful.

The next commentator, Jasmine Byrne from UNICEF who will comment on the question of youth and children and the indicators.  She's Child Protection Specialist leading the UNICEF office on Research, and focuses on Children Rights in a Digital Age and leads Global Kids Online Initiative, 20 years of international experience managing the issues of Child Rights protection programs, et cetera.

>> JASMINE BYRNE: Thank you, Guy.

I want to echo my colleagues speaking before me in congratulating you on this document.  I'm happy to see the issues included of children, both as a cross‑cutting issue, but I'll speak about it separately.  A couple of comments, it would be great to separate children and youth, they're different age groups, Children Rights are distinctly guaranteed by the Convention on right of the child and I'm sure many youth wouldn't like to be lumped together in with children in the same category.  It’s good to see the age desegregation, I know it was mentioned that it is there, but a bit stronger in other sections, not only in a cross‑cutting section on children and youth, particularly when it comes to issues of right of privacy, right to access information, right to freedom of assembly.  They'll have different implications when it comes to children in particular.

Maybe we need qualitative indicators to capture the differences.

It is encouraging that you talk about tracking barriers.  I have some of these copies here.  For children, the barriers may be different barriers than adults face.  In Ghana, for example, there is a global kids online showing 20% of kids without access, there is 68% of them, it is due to parental restrictions.  That's something that we need to take into account and unpack these barriers when you are doing the indicators.  Also to echo what Dorothy was saying, surveys are erratic, they don't happen all of the time.  Even the global kids online happening in 15 countries, it is not going to happen every year.  How do we make sure that we regenerate the statistics?  On the issues related to children, the indicators are mentioned measuring education needs that are looking to affective, safe use, I would just say that they need to be broadened to include all aspects of digital literacy, safety skills, comprehension skills, social and emotional skills and curation skills as you see through the research, a small percentage of children actually make the use of the more complex digital skills.  Even in countries where we consider them digital natives, they're not necessarily digital illiterate.

Finally I would like to say that it would propose an indicator that measures how much children and youth are engaged in the development of policies either through the direct participation or through representation through other agencies who work with them and represent their interests.  I'll be happy to share more comments with David and others in writing as well.

>> GUY BERGER: Interesting.

I thank you so much.  I think that's ‑‑ actually your engagement over the years has made us more sensitive to the issues.  We're not there yet, but thank you for the comments.

Our next speaker is Marten Scalpa who works for the ITU in statistics ‑‑ I don't know the exact title, I hope you will tell us.  I know him from before, you used to work for the UNESCO Institute of Statistics, and he's a great supporter of us trying to develop the national at a statistic commission capacity if you look at media statistics.  I'm sure you will give us some ideas from an ITU point of view but from a national statistics caps tip point of view of engagement with the indicators.  Give us your current title.  I'm sorry.  I didn't have that.

>> MARTEN SCALPA:  Thank you.  I'm senior IT analyst now. 

Indeed, we were planning to go into the nitty‑gritty of statistics and data collection.  You made a few points that were excellent, and I want to tag along on for the breakdown, so many important things to say.

At ITU we collect two types of data:  Telecommunication data that looks at supply side of ICTs, they really look at access and uptake of it.  It we collect household data then, it is on the demand side.  That's where it gets interesting and challenging.

We have seen that in the current list there are very many indicators and I would like to give a warning about the statistical reality of this, especially for the capacity especially Developing Countries, it was often very low.  A lot of indicators, we actually don't need surveys, there is data sources that can be used especially the future because we're not really there yet.  When you want to look at issues of gender, age, you need the socioeconomic breakdowns and for the moment, you only get them through surveys and that's hard to do, costly to do and many countries don't have regular services in place.  This is important points in the physical capacity through the indicators.

The point related to that, it is the methodology, even if it is going to be country assessments, you want to make sure you have a comparison of what's happening in other countries and make sure that countries are using the methodology and it comes back again to the physical capacity issue.  The countries need to understand the methodology and work together with the international organizations, such as ITU and also UIS and others for other indicators and this requires a big effort.

I do see this as a very useful template, although I think there are indicators and they really need to be cut down, maybe even substantially.  I have seen the work on media development indicators and the work you're doing there, going through a country, work with a country, let's say fill in the template that's very useful.

It will still be challenging on many indicators that require access.  I want to finish by saying one other thing.  We're ‑‑ ITU and UNESCO, we work with other organizations, we are part of the member of partnership administering ITU with development, and within that partnership we're looking in developing a dramatic list of indicators of how ICT contributes to the SDGs and which indicators can be used to monitor the progress in that.

This is kind of complimentary to your work.

It will certainly use a lot of the same indicators.  We go also into other areas.

Dave is working with us on that as well.

We'll certainly look at the indicators.  It maybe also is where we can make cross‑cutting remarks there.

Thank you.

>> GUY BERGER: Thank you so much.  We look forward to that continued work.

Our next final input from our guest, we asked specifically to prepare, it is Mr. Raul Echeberria.  I want to tell you something about this man.

Actually he's vice President for global engagement at ISOC but he goes back a long way because he was one of the first to lead the project Connect the Internet network in Uruguay outside of the universities.  I'm not commenting on the age, he's younger than me.  This shows you the background that he has.  He actually got interested in this idea of indicators for the Internet story with UNESCO way back before we even started this particular project because he helped us work with CGI.br and others to see what was the other research on the Internet.  We were able to map.  The whole idea of this is not to duplicate, not to replace, but to see how we can complement, piggy back, accommodate, so on, so Raul was helpful in getting that first phase research done.

As I said, a group that supports the current phase, it is ISOC, which he's, as I said, the Vice President of global engagement and CENA.  Thank you for the support of this thing.  I think that clearly this elephant ‑‑ if we say that the elephant has a stomach, it also has a heart, and I would say that you're part of the beating heart of the elephant with the four legs that we measure to try to move the elephant forward.

>> RAUL ECHEBERRIA: Thank you for your kind words, Guy.  I have to think of the elephant.  If that's something ‑‑

I think that's one of the beauties of this work, that you have done, it is that this integrated approach, the ROAM approach to have all of the pieces together.  We have seen some of those pieces separately, but now we have a model for dealing with all the things in rights, multistakeholder approach, change of things and the openness and all of the other things.  If you look at the Right, I planned to speak about that, but David did already when he showed the example, the category and the themes and Examples of all of the areas.  You can appreciate the variety of the indicators that I think that cover almost everything.

I think this is great.  The value of this is in the implementation.  Some people will use the whole set of indicators, others will use subsets of them.  That's also one of the interesting things of this, that due to the variety of indicators that you have different options for using them.

I love the inclusion of the multistakeholder component here.  Obviously this is a motivation for the countries, for making improvements, for gathering data, measuring things, but also for improving against themselves and against the average of the community and I think we all agree that that is one of the challenges for ‑‑ for making progresses on the Internet Governance in general, it is implementing the multistakeholder approaches at the local level and this will bring the topic to the attention of the governments by including the component.

One of the challenges I see here, it is that we will see amp you run the pilot, that there are some indicators that will not work for every government.  We would see that at some point.

I have realized that some things, for example, in access, in page 17, is there a legal right to access the Internet and online services, this is ‑‑ I know I could put many examples of countries where all the stakeholders, government, they're active in promoting universal access.  I don't know about the existence of a legal right.  It doesn't mean that it is something that prevents the stakeholders and government to take that seriously.  We have probably some countries that will raise what they could find contradictions.  Our services are available for citizens to access and use the languages online, we have to be conscience that in some cases this is not up to a local community and local governments because some languages are very difficult to be coded and the countries, the governments, they're in some cases ‑‑ I could use some examples but I'll avoid doing that to avoid mistakes ‑‑ in some cases the governments and local communities, they're working hard for making those available, the scripts, but it is not possible yet.  I think we should probably make sure the actions that they're taking of if this is a priority for the country into the availability of those services.

Same with multistakeholder participation.  I think we would raise the point that there is the C3 about participation in ICANN.  I think it should be more ‑‑ not only about ICANN but about other ‑‑ I don't think that's ‑‑ it is important to participate in the governmental advisory Committee but it is important that they participate in many other environments, in the regional conferences, on the Information Society, Internet related issues, other ‑‑ and some of the data, I think I didn't read in detail all of the indicators, some indicators, for example this shows how some of the data could be difficult to obtain.  For example, membership of an active participation in working groups, other fora, in some cases, it is very disperse and especially for medium, big countries, it could be difficult to obtain all the information on who is participating because sometimes we see that some people are participating here but they don't know each other, they don't know about the participation of the others.  I think this is an example of how some of the indicators could be difficult to measure.  I think we'll see all of this in detail after the pilot where we get the information about the countries and how difficult it has been for them to self‑assess.

Thank you very much.

As my colleagues said recently, I admire the work you have done.  This is great.  Internet Society is one of the organizations that's looking forward to use those indicators.  Congratulations and a big thank you for the work that you have done.

>> GUY BERGER: Thank you to you too.

I thank all the panelists.  Let's take a bit of response here.  Let's also see if we have any remote questions.  Then we'll ask David to make a response and ask other colleagues if they have a final statement.

We have 25 minutes.  That's good time.

We have somebody in the front there, back there, somebody there.  Let's ‑‑ 3 or 4, let's ‑‑ and 5.  Okay.  Let's take the 5 people and all of them will take remote.  We'll take another round of questions.

So I think we started with the man way down there.  Yes.  Please.

>> AUDIENCE: Tom McKensey, I work with a consultant firm in Paris.  This is fascinating work that you're doing.

I look forward to the final report that you'll be producing.  The question I have regarding the format of the information that you're going to produce, this data that you're producing, how easily ‑‑ how easy it will be for end users of this information to cross‑reference it, to take different indicators and to see ‑‑ so that they can track what's of particular interest to them.

So there are very ‑‑ there are very good ways of doing that.  The whole open data movement, there are good Examples at the OECD with the statistical information, made a lot of statistical information available to a large number of people who we can then process as we like.

Is that your intention with this statistical data in which case I would point it would be extremely useful?

>> GUY BERGER: Thank you for the comment.

This gentleman here.

>> AUDIENCE: Thank you very much.

I came from Bangladesh and as an ISOC Ambassador, I'm a blind person.  I'm not seeing any of you.

This report I think should get in accessible format, if you have some it in accessible formats, accessible PDF, other things, please forward it to me and then I can read.

What I understand, a bigger challenge that is 1 billion people are ignored to receive accessible information to Internet.  We found this is a big disparity.  In my country also we have our government running 25,000 websites.  We're not finding a single website which are accessible for People with Disabilities.  It is improving but creating issues for people with disabilities.  If you look at the ‑‑ how the study will focus on the disability issue and people with disabilities, the concept nothing about us without us, how we're engaging them throughout your study and if there is a scope, we're very happy to collaborate with you.

Thank you.

>> GUY BERGER: Thank you.

Who is number three?  Right.  Over here.

>> AUDIENCE: I work at CGI, I'm a part of the advisory team.  I would like to congratulate the initiative and the work done.

I have two comments on many of this.  This is the first time I'm getting in touch with the work.

I'll concentrate on Internet Governance, multistakeholderism indicators.  I don't know what's behind the indicators, I think they're quite government driven and Internet Governance, multistakeholderism has not been led by government, on the contrary, government is running after Internet Governance and I think we should look at this carefully.  Otherwise, the result on the indicators would be how government support Internet Governance in countries and in this, it has nothing to do with ‑‑ it ‑‑s Internet history, it is very important to look carefully on that.

I totally agree with the comment of Raul, that's important to include other Internet Governance ecosystem and governments other than ICANN.  ICANN is very important, but we have ‑‑ this is important to understand how the community, the countries are participating in this and where the Internet protocols are being produced and if we're talking about Human Rights by design, if you're not participating in an ETF, how can we reach for something so important like that.

Finally, my colleague, the colleague behind me has made a point on that, but the term accessibility refers to the design of the of technologies.  The design of the protocols and has nothing to do with how far Internet access has reached the countries and I was looking for accessibility indicators and the session on accessibility, I couldn't find anyone in that ‑‑ it is good our friend here has brought that issue.  You change the name accessibility, you include the accessibility indicators module for debating it.  It is so important and we ‑‑ we have worked on the accessibility on the web, so I have more things on ‑‑ on indicators here.  I will start participating if you would like to ‑‑

>> GUY BERGER: Please leave that.

>> AUDIENCE: I'll give you after my card.  Then you can talk to me later a bit more.

>> GUY BERGER: Thank you.

>> AUDIENCE: Edmond here from DotAsia.  We're excited about the development that's building on ‑‑ we're launching what we call a youth mobility index within which we define what we call digital mobility and a lot of ‑‑ there is a lot of overlap on some of the indicators here and actually Guy and I talked about it for the last couple of months.  We see a lot of overlap, which is great.  A thing that was mentioned, this is not intentioned to be a comparative study.  I understand that that is specifically for the framework.  I guess a question is, we're interested in, you know, making ‑‑ doing some quantitative, pulling out quantitative analysis, components of it and doing comparative studies like, for example, the different countries, localities around Asia.  Some of the elements in the indicators also indicate that you should look at how the country compares with the global average, as some other countries as well.

I guess my question whether you see there are barriers or that it shouldn't be used at all.  I understand the intent of the particular framework is not for but is there a reason why it cannot be and that is one.

Two specific comments, one, it is interesting that there is no particular component on ‑‑ not a strong component like on the commercial side of things, eCommerce data, things like payment, financial side of things, accept for the very overarching items on GDP, those kinds of things.  That could be interesting.

It is not very prominent.  That's one thing.

The other thing, in terms of the Internet penetration, one thing interesting, you focused on the cost and the coverage, but what about the speed?  It is a bit of a component of the speed in terms of cost and coverage as well.  I think speed would be interesting.  With that, I guess again, going back to ‑‑ I think this is a framework that works very well with what we're doing with digital mobility and youth mobility index that we're coming up with scores and in comparative, I would love to get a sense that this is also ‑‑ we're not breaking the ROAM framework by utilizing the indicators in comparative studies.

A final point, I think what Raul said, without the pilots, we can't breakdown the data and some pilots will pave the way on how we actually collect the data.

Thank you.

>> GUY BERGER: Let's take online, remote question.

>> AUDIENCE: I have a total of 6 or 7 online participants following us.  A quick question, I'm not sure which category they were asking, perhaps I should preface it, is it possible for individuals to have their own database in order to control in the future which companies or other individuals could have access?

>> GUY BERGER: Let's take a second round ever comments, questions.  The gentleman over there, the next gentlemen over there, over here, over there.

Anything, anybody else?  We'll have to close the list then.  Okay.

>> AUDIENCE: Dennis. 

It is particularly interesting in the context of this exercise, and I want to congratulate for the work that you've been doing. 

When I think of the Pacific Island States, Small Island States, these are only a couple of million people, but there are many of the UNESCO Member States opening statement.  If I look at the indicators, I feel that many of those could probably not be readily collected.  I'm wondering your take on this particularly with regard with capacities of statistical offices and how the collection can be arranged.  Then it is with regard to some of the indicators, particularly those that measure broadband fixed lines, that obviously is something that doesn't apply to the geography of some of the small island states where a fixed line doesn't necessarily make sense and a mobile line makes more sense.  The question of whether there is alternative indicators for some states where this doesn't make so much sense.

>> GUY BERGER: Thank you.

>> AUDIENCE: Thank you very much.  I'm from Tanzania.

Indicators for people with disabilities should be extensively researched.  There are a different kind of disabilities apart from those that can't see, but there are some people with disabilities that they don't have legs, they don't have hands, how do they access the Internet?  An indicator should be really put over there.

Number two, indicators about countries that are very ‑‑ we see a tendency now for countries, sometimes in the past, they were free, but now the freedom, it has been curtailed.  We have to be very careful in how we involve the countries in coming up with indicators.  You can see now a country like Tanzania, which was a little bit free, but we have a new President would is quite very dictorial, they're limiting people in accessing Internet, we have a very bit very careful on how we prepare our indicators.

Thank you.

>> GUY BERGER: Thank you.

>> AUDIENCE: I'm a member of the advisory group on the project.

So ‑‑ of course we ‑‑ I join all colleagues saying that the project is fantastic, ambitious.  Let me also add that as a great added value, that the project, it looks not only at the access and the quality of access, but probably one of the first things to look at, the quality of the contributions to date and the capacity of people to shape the Internet, to produce contents online.  That's ‑‑ it should be obvious because we ‑‑ that's why we like the Internet.  It is usually undermeasured and underassessed.  Because of that, probably I agree that we have lots of indicators, and probably the next phase would be the main challenge, it would be to simplify the way we assess the Internet universality, but we need probably ‑‑ my advice is, and we have other moments to discuss this more, but it would be to work on the indicators measuring the content production.

Here I see for example a registered domain, but probably this data will be always falsified, probably someone from ICANN can give us a dimensions of cybersquatting.  In many areas worldwide we have a high number of reduced domain not telling us anything about the actual contents online.

There are, of course, many other ways to measure online contents and in some cases we need to ask how the Internet service providers, the Internet intermediaries, this goes with ‑‑ another comment, a suggestion that I have, it would be probably difficult for countries to collect that kind of data.  It would be good for them to find the agreement or the collaborations with the big Internet service providers.

It might be the case that some of the indicators, especially when talking about the quantitative indicators, it would be collected in a centralized way by UNESCO I hope.  That would be also important because you're going to secure the collection of data will happen in a vigorous way.  It will be pretty easy work to do.  You will collect the data by reliable sources, of course.  Then eventually you leave all of the qualitative parts of the collection of data when you look at the regulatory and legal aspects of the Internet indicators to the country.

In this way, we can give an incentive to local actors.  You will say UNESCO has collected a bunch of data.  Help us to feed this from your side.

>> GUY BERGER: I have to ask the colleagues to be short.  Now we're beginning to run out of time.

>> AUDIENCE: Hello.  Thank you so much for your tremendous job and as you also mentioned here, we're really taking this questionnaire, we're sharing it among our young people in the Civil Society in Russia.  I represent the Russian NGO and involved in global governance and I'm interested in such indicators as multistakeholder and I believe there should be distribution of responsibilities so that there is no abuse of power in any country or community.  I believe that Internet Governance should really be founded in the three‑level model with international organizations, Technical Community, governmental level and Civil Society level.  I believe this should be really a strongly voiced item in the indicators.  What's probably needed, the U.N. legal framework, the Convention where an indicator can be among the cross‑cutting indicators.

Also what I mentioned from the 0 day, I strongly believe that such indicator as security, you can put it on page 29, it is group D, yes, trust and security, but broadly I think that security has a potential of being a separate indicator.

If you even look to this program, yes, of the event, it is all about cybersecurity and threats and so that's a big threat we have.  It is a remark to the indicators.

My question is, how to get involved in testing the indicators, how we as Civil Society can be helpful, what about expert groups can join.  Tell us.  We're very excited.

>> GUY BERGER: Thank you very much.

>> AUDIENCE: I work for ICANN.

I have a comment and a question.  I'm happy to see that UNESCO doesn't shy away from challenges.

This is clearly a challenge.  A good one.

My comment, as a network of networks, technical definition, it is borderless.  Many indicators are based on a national data gathering.  I'm trying to see how you can combine the two things and different layers in that.  Here we don't define the Internet but some are in the application level, infrastructure level, some in the participation level, and those different levels, they also have components and data that are completely independent from one country or a single stakeholder group.

Eye

The question is, whether the fragmentation part of the Internet is within the Internet universality.

What I see here, probably one of the measures to date that we're seeing, it is that the global network here, it is becoming more and more compartmentalized at the technical level, the application level, it is the Internet we know only five years ago, ten years ago, whether there is universality indicators, also looking at the dimensions of how you can do without.

The last comment I have, it is probably more of the decision that could lead to this, it is the time‑sensor sensitive of the data gathering.  Many of the indicators, they change by probably the second and you take a snapshot of those data.  Maybe 5 minutes after things may be influenced.  How do you combine that?

>> GUY BERGER: We're really running out of time. 

The gender imbalance and questions, there are two women, I think I have to sacrifice the comments from my panelists here to get the two comments from the two women in the audience.  You indicated first and then you, please, very short, we have basically one minute left!

>> AUDIENCE: I would like to congratulate UNESCO on this excellent set of indicators.  The product is incredibly interesting, even while it is still in progress. 

My question regards to the expected follow‑up.  I understand that the indicators will be presented formally in September next year and my question is, what is the expected follow‑up, in which way would UNESCO like to plan to encourage states to elaborate actually national reports and gather data exchange of information.  As you know, because of you elaborating on the own set of freedom indicators, in 2016 and we have the national reports, but I'm sure that you're aware of this.  So we observe the crucial information tool to engage in the supporting implementation.  I would like to hear from you about it.

>> Can you repeat which country?

>> AUDIENCE: Council of Europe.

>> GUY BERGER: Yes.

>> Hello.  I'm looking at it from the operability perspective, I'm not sure this is covered here and may it is covered somewhere else.  Just ‑‑ I'm just asking, for example, if we can make some kind of law which every country can collaborate and help each other to share.  For example, environment, it is so many disasters, national disasters, pandemic occurrence and then there is also the basic bottom line is ‑‑ also things, if you have a dataset and then everybody in the world, you can observe that and then when they have seen the ‑‑ is it the ‑‑ out of the range of study?  Is it ‑‑

>> Those local people's understanding, the data, the local as well as the international data, if we can share on that also seeing an error, it could be ‑‑ there is some kind of a gap between.  It draws knowledge as we're together with national security safety not only attacking or negative side, but knowing and helping each other can create a real positive side in terms of the health collaboration and if your idea is to contribute somewhere else the role.

>> GUY BERGER: Clearly we can't respond to all of the questions.  We now have to stop our workshop.  I could say two things:  One, please put your points on the website.  We I hope in about 10 days have an interactive interface.  You can send a little memo if you want.  We would love to continue working with you, if you want us to keep in touch, please put your email there.  We'll not spare you, but we'll keep you informed on this.  We really value the questions.

This still has some way to go for the further comments and the pretesting and the piloting.  We hope in 2019 it is really, really ready to roll and make use of all or part of it.  If you watch the website, you'll see different ways to continue to be involved in input.  We think that these indicators can really represent the global wisdom and be useful to the maximum number of stakeholders.

I think I would like to ‑‑

>> Sorry to block your lunch time.

We have ISOC from Nigeria.  She said I would like to say that in gathering data, emphasis should be made on extracting information about the actual field, feedings of the user base.  It is the one thing a tech said in providing so much speed to the Internet ‑‑ it is another thing to have the fire flies with the user and of course, the broad Spectrum of locations.

>> GUY BERGER: Right.

Thank you.

That's my colleague who has put together this panel.  I thank you very much.

The other colleague ‑‑ is he there?  Yeah.  He had to leave?  Okay.

>> Yeah.  Our colleagues in the Geneva office, yeah.  They wanted to share with that.

>> GUY BERGER: I wanted to thank them and you all. 

This is important, one of my colleagues here, David ‑‑ not colleague, but the consultant has taken notes, and he's one short response.  Maybe I just think ‑‑ we welcome as much input as you can possibly give. 

Just to add a couple of things:  There are no perfect questions, indicators, no perfect answers.  We shouldn't search for perfection, but what is feasible here to do in majority of the countries.  Similarly, we can't be comprehensive, we have to focus on things that will give us the evidence we need to assess the national, international environments collectively.

I think we can achieve something that's really useful as the media developer indicators have been in the past.

>> PAUL FEHLINGER: Accompany us and at the next IGF, we'll have another workshop, you can be updated further and you'll see how we improved from today.  I value the participation and I look forward to the outcomes of the indicators.

Thank you and have a good lunch.