- 07 Oct, 2020 1 commit
-
-
Simone Basso authored
Part of https://github.com/ooni/probe-engine/issues/668
-
- 06 Oct, 2020 4 commits
-
-
Simone Basso authored
Release 0.18.0 cycle
-
Simone Basso authored
-
Simone Basso authored
-
Simone Basso authored
See https://github.com/ooni/explorer/issues/495#issuecomment-704101604 While there, add a QA script for making sure everything is WAI. Interestingly this led to discover a minor issue where the network name was not included when empty.
-
- 05 Oct, 2020 9 commits
-
-
Simone Basso authored
With https://github.com/ooni/probe-engine/pull/966 and this change together we have a more in depth mechanism for enforcing that we're doing the right thing in terms of honouring sharing settings. Yet, my instinct is that we should push for removing this vestigial options that increase complexity and reduce the quality of data we receive, in the mid term. Shall discuss this with the rest of the team.
-
Divyank Katira authored
Co-authored-by:
Simone Basso <bassosimone@gmail.com>
-
Simone Basso authored
With 7ffe9dc4, we introduced privacy preserving geolocation accessors. With this commit, we start using them. It turns out the original problem that triggered this investigation (i.e., https://github.com/ooni/explorer/issues/495) happens because the code inside experiment.go sends the real ASN when opening the report. This is not the correct behavior, which is to access the ASN through the the privacy preserving wrapper. Such that, if the user does not want to share their ASN with OONI, the report is opened using "AS0". Upon inspection, it seems we need to apply the same fix also to other fields of the openReport and Measurement structures. And, even if we were not needing that, it seems the most correct thing to do anyway. This diff does precisely that. It ensures we always use the new accessors and adds a bunch of new tests to ensure this is WAI.
-
Simone Basso authored
This change extracts from experiment.go the ExperimentBuilder struct as well as the list of all the experiments. It makes working with experiment.go more manageable by moving separate components into two distinct separate files. This change is yak shaving while I am investigating https://github.com/ooni/explorer/issues/495.
-
Simone Basso authored
The previous diff (https://github.com/ooni/probe-engine/pull/963) has introduced the concept of an accessor for the resolver IP whose returned value changes depending on the configured privacy settings. This is a good programming practice. So, while I am slowly approaching to the bottom of https://github.com/ooni/explorer/issues/495, I though it was good yak shaving to implement accessors for every variable for which the configured privacy settings matter.
-
Simone Basso authored
Since this experiment has been rewritten in Go, we can safely omit the IP address of the client resolver. The new implementation, in fact, performs a geolookup of such an IP address. Not strictly and directly related with the issue I am working on (https://github.com/ooni/explorer/issues/495). Yet, working on such an issue and reading the code is a useful mindset to reason about simplifications around what we submit to our backends.
-
Simone Basso authored
This concludes our initial yak shaving journey. Now the view of a Session that experiments have is significantly simpler than before. I see this diff as a preliminary change aimed at facilitating investigating https://github.com/ooni/explorer/issues/495 in the probe context.
-
Simone Basso authored
This continues the work initiated in https://github.com/ooni/probe-engine/pull/960 and the rationale is also of course the same. This work is indeed yak shaving while looking into https://github.com/ooni/explorer/issues/495.
-
Simone Basso authored
ExperimentSession is a large interface in the model package that is too large and too widely used to be easily modifiable. I noticed this issue when I started investigating the issue at https://github.com/ooni/explorer/issues/495. Before we embark into more substantial changes, let us clear the table a little bit by reducing the scope of ExperimentSession.
-
- 30 Sep, 2020 2 commits
-
-
Simone Basso authored
-
Simone Basso authored
* README.md: remove release procedure Better version at https://github.com/ooni/probe-engine/blob/master/.github/ISSUE_TEMPLATE/routine-sprint-releases.md * Update direct and indirect dependencies * Update internal/httpheader/useragent.go Just choose the currently most popular header. * We're now at version 0.17.0 * routine-sprint-releases.md: remember to declare alpha version * resources/assets.go: use ooni/probe-assets@20200929203018 * netx/gocertifi/certifi.go: we already had latest CA * legacy: disable tests that often fail in GHA These tests pass normally when run locally but fail quite often when run from GitHub Actions (GHA). So disable them when inside GHA. See https://github.com/ooni/probe-engine/runs/1184212564
-
- 29 Sep, 2020 2 commits
-
-
-
Simone Basso authored
* oonimkall/session.go: significantly simplify the implementation When helping @lorenzoPrimi with https://github.com/ooni/probe-android/pull/360 I had the impression there was too much complexity. Then we said we could improve the situation by changing the class names as written in https://github.com/ooni/probe-engine/issues/951#issue-708184925. But then there was still too much complexity. I could not easily picture in my mind why each class was needed. This was actually a strong signal that we needed to refactor the code more than just renaming clases. So, here's the new approach (for which I'll write soon a mini design document that will be referenced inside the code - or maybe this is the document!): 1. Session uses runtime.SetFinalizer therefore we don't need to be concerned about wrapping it and using AutoCloseable in Java. It is not normal for Go code to use finalizers, but I believe it makes sense in this context, _because_ it simplifies interfacing Go and Java code. 2. We introduce again a Context instance _because_ it was too painful to have each Task define the same operations and also too bloated. 3. A Context is created from a Session, therefore the Session can keep track of what `cancel()` functions to be called when done. So now the programmer is feee to just not be concerned about calling `cancel()` _unless_ there is the need for really cancelling the operation. The Session will specifically call all the cancelling functions inside of its finalizer. 4. All Session methods are now "synchronized" with respect to a mutex that is inside of the session itself. This removes any concerns regarding the usage from multiple threads. Yet, we're still not able to have a long running session yet, mainly because now we're tracking child contexts. But I also do not think pushing for a global session is particularly useful. 5. The resubmission now happens within a session. We have a submitter inside the session and we are confident thread-safe-wise there are no issues since also the method for submitting is, of course, thread safe. This strategy completely eliminates the need of wrappers for the sake of introducing AutoCloseable properties in the code. While there, I also modified `/session.go` such that it's possible to perform an unmemoised lookup of the probe location. This has been instrumental to remove one more limitation of the previous `/oonimkall/session.go` API. * oonimkall: improve new session API's tests This ensures that we're cleaning up everything using finalizers. While there are debugging code to understand flaky test. * final changes
-
- 24 Sep, 2020 1 commit
-
-
Simone Basso authored
Co-authored-by:
xhdix <xhdix@yahoo.com>
-
- 21 Sep, 2020 3 commits
-
-
Simone Basso authored
* build-cli.sh: say it when we apply workaround I just figured it's probably more transparent to inform the user running the build script that we're applying this workaround. Also, in readme_compiletimecheck.go, let us mention explicitly what is the build script that we could use for increased clarity. Related to https://github.com/ooni/probe-engine/issues/866. * github/workflows: make sure we don't break go1.15 builds We now have a workaround for people building using go1.15. We wanna make sure with a simple smoke test we produce a working miniooni binary. Also part of https://github.com/ooni/probe-engine/issues/866.
-
cyBerta authored
* workaround to build probe-engine with go1.15 (#866) * change order of arguments so that empty -tags argument will be ignored correctly
-
Simone Basso authored
Push is not triggered for external pull requests and, at the same time, it makes sense we run more thorough checks when we have a PR. Noticed when reviewing https://github.com/ooni/probe-engine/pull/944.
-
- 17 Sep, 2020 1 commit
-
-
Simone Basso authored
There are two reasons for this change: 1. we wanna make sure ams-pg.ooni.org passes all tests 2. mia-ps-test.ooni.io is currently broken See https://github.com/ooni/backend/issues/446
-
- 15 Sep, 2020 3 commits
-
-
-
Simone Basso authored
It didn't occur to me initially, but the fact that vim adds a newline at the end of the file adds an extra blank input to be tested. We actually don't want to emit an error for such an input and the fix is just to avoid adding blank inputs to the inputs list. Since the fix is so easy, while there allow any other blank line in the middle of the file, since it costs us nothing. Part of https://github.com/ooni/probe-engine/issues/926
-
Divyank Katira authored
-
- 11 Sep, 2020 4 commits
-
-
Simone Basso authored
* Revert the patch because it's causing issues when using miekg/dns * Make sure we run tests for external PRs
-
Simone Basso authored
This reverts commit 83089d6e because it turns out we don't correctly support IDNs in all cases. See ``` 2020/09/11 23:33:17 debug resolve dns.google.com... 2020/09/11 23:33:17 debug resolve dns.google.com... ([8.8.4.4 8.8.8.8 2001:4860:4860::8888 2001:4860:4860::8844], <nil>) in 44.150342ms 2020/09/11 23:33:17 debug resolve яндекс.рф... 2020/09/11 23:33:17 debug resolve яндекс.рф... ([77.88.55.66 5.255.255.55 5.255.255.5 77.88.55.55 2a02:6b8:a::a], <nil>) in 1.852079ms 2020/09/11 23:33:17 debug resolve dns.google.com... 2020/09/11 23:33:17 debug resolve dns.google.com... ([8.8.4.4 8.8.8.8 2001:4860:4860::8844 2001:4860:4860::8888], <nil>) in 73.81118ms 2020/09/11 23:33:17 debug resolve яндекс.рф... 2020/09/11 23:33:17 debug resolve яндекс.рф... ([], dns: domain must be fully qualified) in 50.781µs --- FAIL: TestIntegrationNewResolverUDPAddress (0.07s) integration_test.go:48: dns: domain must be fully qualified ``` We need to work for more time on this patch.
-
xhdix authored
-
- 09 Sep, 2020 2 commits
-
-
Simone Basso authored
We've added new APIs in https://github.com/ooni/probe-engine/issues/893 but we didn't feel the need of updating DESIGN.md, rather it was more productive to use an issue as a designing tool. This is all fine and we should do whatever works(tm) but we should also make sure we don't provide false expectations with DESIGN.md. A cheap way to do that is to clearly mark this design document scope to be that of describing how to replace Measurement Kit.
-
Simone Basso authored
Part of finishing https://github.com/ooni/probe-engine/issues/893 and of ensuring the Session API we're exposing is compliant with the compact design at https://github.com/ooni/probe-engine/issues/893#issuecomment-689031449.
-
- 08 Sep, 2020 1 commit
-
-
Simone Basso authored
See https://github.com/ooni/probe-engine/issues/893#issuecomment-689031449 for context on why we needed to rewrite the API.
-
- 07 Sep, 2020 1 commit
-
-
Simone Basso authored
* oonimkall/session.go: remove Context.Close We don't need to have a method matching AutoCloseable directly here since we can create a wrapper when we need it (Android). See https://github.com/ooni/probe-engine/issues/893#issuecomment-685688006. * oonimkall/session.go: Context => TaskContext Avoid gratuituously conflicting with well known classes that have the same name and another meaning both in Go and Android. See https://github.com/ooni/probe-engine/issues/893#issuecomment-685688006 * oonimkall/session.go: use plural and consistent naming See https://github.com/ooni/probe-engine/issues/893#issuecomment-685688006 (These inconsistencies emerged when trying to integrate 0.17.0-beta with the existing Android codebase.)
-
- 04 Sep, 2020 4 commits
-
-
Simone Basso authored
With the version bump, we're ready to merge into mobile-staging so that we can start testing the new code with Android and iOS. Part of https://github.com/ooni/probe-engine/issues/893
-
Simone Basso authored
The field is deprecated but not using it is going to change the resubmitter behavior wrt MK's submitter. So, let's be pragmatic and add this field again. See https://github.com/ooni/probe-engine/issues/893#issuecomment-686973455
-
Simone Basso authored
With the availability of the SubmitTask, we now have code to which we can pass measurements to resubmit without being concerned of whether they belong to the same report or not. The code will indeed to the right thing and we will open/close reports as needed. Design at https://github.com/ooni/probe-engine/issues/893#issuecomment-686613123
-
Simone Basso authored
The submitter will take in input measurements and do the right thing with respect to using the current report or open a new one. With this functionality implemented, we have now all the building blocks required by https://github.com/ooni/probe-engine/issues/893.
-
- 03 Sep, 2020 2 commits
-
-
Simone Basso authored
The criteria is: if the report template matches, there is no point in opening a new report. Otherwise, yes, they are different reports. Needed by https://github.com/ooni/probe-engine/issues/893
-
Simone Basso authored
Part of https://github.com/ooni/probe-engine/issues/893
-