16:59:52 <hellais> #startmeeting
16:59:52 <MeetBot> Meeting started Mon Apr  4 16:59:52 2016 UTC.  The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:59:52 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
17:00:11 * willscott waves
17:00:22 <hellais> so who is here?
17:00:25 <landers> here
17:00:31 <anadahz> hello
17:00:38 <poly> hey all
17:00:51 <agrabeli> hello
17:01:46 <hellais> excellent, so what have you been up to last week?
17:03:05 <bnvk> hellpo
17:03:20 <landers> nothing ooni-related by me
17:03:41 * sbs here!
17:04:02 <poly> Alright, so I've been working with hellais on the ooni API, which we've decided to rename "HTTP Control Protocol". I spent a little while fleshing out a fuller specification, which you can find here https://github.com/0xPoly/ooni-probe/blob/HTTP_Interface/docs/source/control_interface.rst
17:04:25 <hellais> I worked on fixing bug in the json_reports branch, started to sketch out the web API to support the GUI of OONI, began adding JSON support to MK and fixed some bugs in ooniprobe.
17:04:49 <poly> We discussed implementation strategies and I believe hellais has started experimenting with some code
17:06:05 <agrabeli> wrote a blog for OONI which will be released at a later stage EOF.
17:06:18 <sbs> I was traveling for part of the week, so was not very effective; still I managed to push some hotfixes and preparatory changes for MK 0.2 and I also completed the review of the JSON pull request for ooni (i.e. #456) which was merged just few minutes ago. EOF
17:07:29 <landers> oh wow, the json reports thing is merged -- nice job
17:08:13 <bnvk> I haven't done anything on OONI since launch, as I was in Poland. I'm curious which issues are most important for me to tackle hellais
17:08:58 <nuke> Hi
17:09:43 <sbs> nuke: hi!
17:11:14 <anadahz> I was at Rightscon working on how to collaborate better with censorship circumvention tools developers; helping us to test and create tests as well as submitting measurements to the pipeline. Many groups and people asked on how they can contribute to OONI and especially how they can contribute to the URL lists (https://github.com/citizenlab/test-lists/pull/35). I gave a short demo of OONI explorer
17:11:16 <anadahz> people and had very good feedback --people like it! EOF
17:12:23 <poly> anadahz: that's great to hear!
17:12:46 <hellais> anadahz: very good!
17:15:18 <hellais> anadahz: any particular takeaways from rightscon worthwhile sharing?
17:16:37 <anadahz> hellais: wrt to OONI explorer people have been asking to get a list or dump of all reports per country
17:17:26 <hellais> anadahz: this is something different than what is available at measurements.ooni.torproject.org?
17:18:56 <anadahz> hellais: yes these measurements^are per date so they were asking how they can get all of them in a per country way
17:20:20 <anadahz> Alkasir has started to collect some reports which they will happily submit to OONI pipeline
17:21:49 <hellais> anadahz: hum I see, well they can list all the reports for a given country from the explorer API and then use the filenames to download the reports.
17:22:07 <hellais> anadahz: I don't think we want to be duplicating the data to support every possible way of clustering them though
17:22:56 <anadahz> hellais: perhaps we could make this on-demand via the API?
17:23:13 <MightyOctopus> [13measurement-kit] 15bassosimone created 06bassosimone-patch-1 (+1 new commit): 02https://git.io/vVEeq
17:23:13 <MightyOctopus> 13measurement-kit/06bassosimone-patch-1 144cc16e4 15Simone Basso: README.md: mention Neubot's MPEG DASH test
17:23:34 <hellais> anadahz: what do you mean by on-demand?
17:23:35 <MightyOctopus> [13measurement-kit] 15bassosimone opened pull request #443: README.md: mention Neubot's MPEG DASH test (06master...06bassosimone-patch-1) 02https://git.io/vVEeG
17:23:45 <willscott> per-country and per-test seem like desirable slices
17:24:21 <willscott> could be an automatic symlink process to give directory views or just a file with the relevant test urls that make it easy to grab those subsets
17:25:12 <anadahz> hellais: like the API should provide a json dump upon a selection of reports
17:25:50 <anadahz> hellais: sort of an export functionality
17:28:34 <anadahz> Alkasir (https://alkasir.com/) has started to collect more blocking measurements reports which they will happily submit to OONI pipeline
17:28:35 <hellais> hum, yeah I guess we could have something like that
17:30:09 <hellais> anadahz: yes I remember exchanging some emails with them about that, though the last time I spoke to them, it didn't seem like they would have time to implement submission to the OONI pipeline, but we would have to write our own integration with their DB
17:30:16 <hellais> anadahz: has that changed?
17:31:21 <anadahz> hellais: yes they have not yet sorted this out due to some dev issues
17:33:04 <anadahz> poly: the HTTP Control Protocol is based on this: https://ooni.readthedocs.org/en/latest/architecture.html#ooniprobe-api ?
17:33:21 <poly> correct
17:33:50 <poly> it's updated to account for decks, which were not a part of OONI when that section was originally written
17:34:35 <poly> moved to to a seperate file to make it easier to find
17:35:44 <anadahz> poly: The Authentication issue in: https://github.com/0xPoly/ooni-probe/blob/HTTP_Interface/docs/source/control_interface.rst#id16 could be solved by making this service available only to localhost
17:37:50 <anadahz> In other news I got quite many questions on how one can install ooniprobe. We should really work on the packaging and testing of ooniprobe.
17:38:10 <poly> anadahz: I discussed this with hellais. For GUI purposes, serving to only is the easiest solution. But we are also considering using the same protocol for controlling ooniprobe on a RPi, so authentication might come in handy in the future
17:39:14 <anadahz> Perhaps it's even possible to run ooniprobe in Windows now: https://blogs.windows.com/buildingapps/2016/03/30/run-bash-on-ubuntu-on-windows/
17:40:48 <willscott> anadahz: while possible, not simple to package or make usable for windows users
17:41:39 <hellais> anadahz: there are many roadblocks to getting ooniprobe running on windows: * Making all scapy based stuff a soft depedency and make it run without them (there is no way of automating the installation in winpcap required to use scapy) * There are quite a few places where we assume the filesystem to follow posix compatible * Having a GUI (without a GUI I don't think it makes sense to even attempt targetting w
17:41:46 <hellais> indows users)
17:42:11 <anadahz> poly: like users controlling the RasPi in their LAN?
17:42:52 <poly> anadahz: sure, or possibly for remote control over the internet is you set it up that way
17:43:06 <anadahz> hellais: willscott indeed but it's a good start
17:43:42 <anadahz> poly: you can share a localhost service over Tor HS
17:44:41 <poly> anadahz: good point, that would make things much easier
17:45:30 <hellais> still even if it's hosted as a tor hidden service it should have authentication
17:45:55 <willscott> hellais: in previous discussions i've been more optimistic about MK being a solution for windows, since c code is relatively portable across platforms
17:47:41 <anadahz> hellais: yes we could cookie based authentication
17:48:00 <hellais> willscott: yeah that is also another option, though MK is still distant from feature parity with ooniprobe and even when the library is complete we would need to put some GUI on top it nonetheless
17:48:04 <poly> either way, I don't think we should worry too much about authentication at this stage
17:49:11 <hellais> yup
17:52:17 <hellais> so I guess we should proceed into the next steps
17:52:23 <MightyOctopus> [13measurement-kit] 15bassosimone closed pull request #443: README.md: mention Neubot's MPEG DASH test (06master...06bassosimone-patch-1) 02https://git.io/vVEeG
17:52:24 <MightyOctopus> [13measurement-kit] 15bassosimone pushed 1 new commit to 06master: 02https://git.io/vVEIR
17:52:24 <MightyOctopus> 13measurement-kit/06master 144a1e12c 15Simone Basso: README.md: mention Neubot's MPEG DASH test
17:54:42 <hellais> bnvk: regarding things to be done I think this would be a very useful addition to the explorer: https://github.com/TheTorProject/ooni-explorer/issues/75
17:54:55 <MightyOctopus> [13measurement-kit] 15bassosimone 04force-pushed 06feature/ndt from 143ee87c6 to 14d370a35: 02https://git.io/vVR0b
17:54:55 <MightyOctopus> 13measurement-kit/06feature/ndt 14d370a35 15Simone Basso: Start working to NDT client implementation
17:57:57 <hellais> my next steps are: make changes to the ooni-pipeline to handle 10s of thousands of small reports (this lead to the reports from the past 3 days not being spooled properly), work on the JSON support in MK and the web connectivity test in ooniprobe
17:58:27 <bnvk> hellais: cool, thanks for focusing that for me!
17:59:09 <poly> I will create a PR with the spec and hopefully start working on implementing it
17:59:35 <poly> still also need to decide on which web framework (if any) to use for the GUI - suggestions are welcome
18:00:30 <sbs> I will work to enable submitting OONI reports using HTTPS and will also work on the NDT client - EOF
18:01:13 <anadahz> next steps: improve ooniprobe packaging and installation, continuing on documenting the OONI explorer EOF
18:01:22 <nuke> Sbs: I like that ;)
18:03:11 <MightyOctopus> [13ooni-probe] 150xPoly opened pull request #473: OONI Control Interface (06master...06HTTP_Interface) 02https://git.io/vVEqT
18:03:53 <sbs> nuke: :)
18:05:38 <hellais> ok well if there is nothing else, that's all folks
18:05:41 <hellais> thanks for attending
18:05:48 <hellais> #endmeeting