mat2-web issueshttps://0xacab.org/jvoisin/mat2-web/-/issues2023-07-09T16:29:53Zhttps://0xacab.org/jvoisin/mat2-web/-/issues/69Inline Cleaning2023-07-09T16:29:53ZjfriedliInline CleaningAutistici developed a stripped down version of mat2-web using only a single endpoint that returns the cleaned file inline.
https://git.autistici.org/noblogs/wp-mat-server
This behaviour can be beneficial as the current endpoint that is...Autistici developed a stripped down version of mat2-web using only a single endpoint that returns the cleaned file inline.
https://git.autistici.org/noblogs/wp-mat-server
This behaviour can be beneficial as the current endpoint that is meant for machine to machine communication still returns a download link. Having the possibility to retrieve the cleaned file might be useful for some implementations.
Should we implement this as well? @jvoisinhttps://0xacab.org/jvoisin/mat2-web/-/issues/68Container contains .git/config2023-03-29T13:02:48ZcasperContainer contains .git/configThe backend exposes it's `.git/config`, including a `gitlab-ci-token`. I guess, this is probably not intentional?The backend exposes it's `.git/config`, including a `gitlab-ci-token`. I guess, this is probably not intentional?https://0xacab.org/jvoisin/mat2-web/-/issues/66Please release 0.8.7 and 0.8.82023-03-14T10:09:26ZgeorgPlease release 0.8.7 and 0.8.80.8.7 and 0.8.8 were tagged three weeks ago, but so far there haven't been any related releases. Is this by accident, or by design?
Thanks for your work!0.8.7 and 0.8.8 were tagged three weeks ago, but so far there haven't been any related releases. Is this by accident, or by design?
Thanks for your work!https://0xacab.org/jvoisin/mat2-web/-/issues/65Fix Seccomp Profile2023-02-24T15:41:20ZjfriedliFix Seccomp ProfileThe current seccomp profile prevents the container from starting
```bash
Attaching to mat2-web-demo_api_1
api_1 | sh: 1: Cannot fork
```The current seccomp profile prevents the container from starting
```bash
Attaching to mat2-web-demo_api_1
api_1 | sh: 1: Cannot fork
```https://0xacab.org/jvoisin/mat2-web/-/issues/64Fix Container2023-02-24T12:00:22ZjfriedliFix ContainerThe :latest container image is broken and cannot start up
```bash
test-mat2-web-api-1 | [uWSGI] getting INI configuration from /etc/uwsgi/apps-enabled/mat2-web.ini
test-mat2-web-api-1 | *** Starting uWSGI 2.0.18-debian (64bit) on [Sat...The :latest container image is broken and cannot start up
```bash
test-mat2-web-api-1 | [uWSGI] getting INI configuration from /etc/uwsgi/apps-enabled/mat2-web.ini
test-mat2-web-api-1 | *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Jan 28 13:43:22 2023] ***
test-mat2-web-api-1 | compiled with version: 8.2.0 on 10 February 2019 02:42:46
test-mat2-web-api-1 | os: Linux-5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023
test-mat2-web-api-1 | nodename: 8271be393be6
test-mat2-web-api-1 | machine: x86_64
test-mat2-web-api-1 | clock source: unix
test-mat2-web-api-1 | pcre jit disabled
test-mat2-web-api-1 | detected number of CPU cores: 8
test-mat2-web-api-1 | current working directory: /var/www/mat2-web
test-mat2-web-api-1 | detected binary path: /usr/bin/uwsgi-core
test-mat2-web-api-1 | chdir() to /var/www/mat2-web/
test-mat2-web-api-1 | your memory page size is 4096 bytes
test-mat2-web-api-1 | *** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
test-mat2-web-api-1 | detected max file descriptor number: 1048576
test-mat2-web-api-1 | lock engine: pthread robust mutexes
test-mat2-web-api-1 | thunder lock: disabled (you can enable it with --thunder-lock)
test-mat2-web-api-1 | uwsgi socket 0 bound to UNIX address /run/uwsgi/uwsgi.sock fd 3
test-mat2-web-api-1 | Python version: 3.7.3 (default, Oct 31 2022, 14:04:00) [GCC 8.3.0]
test-mat2-web-api-1 | *** Python threads support is disabled. You can enable it with --enable-threads ***
test-mat2-web-api-1 | Python main interpreter initialized at 0x56372419e020
test-mat2-web-api-1 | your server socket listen backlog is limited to 100 connections
test-mat2-web-api-1 | your mercy for graceful operations on workers is 60 seconds
test-mat2-web-api-1 | mapped 145840 bytes (142 KB) for 1 cores
test-mat2-web-api-1 | *** Operational MODE: single process ***
test-mat2-web-api-1 | Traceback (most recent call last):
test-mat2-web-api-1 | File "./main.py", line 5, in <module>
test-mat2-web-api-1 | from matweb import utils, rest_api, frontend
test-mat2-web-api-1 | File "./matweb/utils.py", line 8, in <module>
test-mat2-web-api-1 | from libmat2 import parser_factory
test-mat2-web-api-1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/__init__.py", line 7, in <module>
test-mat2-web-api-1 | from . import exiftool, video
test-mat2-web-api-1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/exiftool.py", line 9, in <module>
test-mat2-web-api-1 | from . import abstract
test-mat2-web-api-1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/abstract.py", line 7, in <module>
test-mat2-web-api-1 | class AbstractParser(abc.ABC):
test-mat2-web-api-1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/abstract.py", line 36, in AbstractParser
test-mat2-web-api-1 | def get_meta(self) -> dict[str, Union[str, dict]]:
test-mat2-web-api-1 | TypeError: 'type' object is not subscriptable
test-mat2-web-api-1 | unable to load app 0 (mountpoint='') (callable not found or import error)
test-mat2-web-api-1 | Traceback (most recent call last):
test-mat2-web-api-1 | File "/var/www/mat2-web/main.py", line 5, in <module>
test-mat2-web-api-1 | from matweb import utils, rest_api, frontend
test-mat2-web-api-1 | File "./matweb/utils.py", line 8, in <module>
test-mat2-web-api-1 | from libmat2 import parser_factory
test-mat2-web-api-1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/__init__.py", line 7, in <module>
test-mat2-web-api-1 | from . import exiftool, video
test-mat2-web-api-1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/exiftool.py", line 9, in <module>
test-mat2-web-api-1 | from . import abstract
test-mat2-web-api-1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/abstract.py", line 7, in <module>
test-mat2-web-api-1 | class AbstractParser(abc.ABC):
test-mat2-web-api-1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/abstract.py", line 36, in AbstractParser
test-mat2-web-api-1 | def get_meta(self) -> dict[str, Union[str, dict]]:
test-mat2-web-api-1 | TypeError: 'type' object is not subscriptable
test-mat2-web-api-1 | unable to load app 0 (mountpoint='') (callable not found or import error)
test-mat2-web-api-1 | *** no app loaded. going in full dynamic mode ***
test-mat2-web-api-1 | *** uWSGI is running in multiple interpreter mode ***
test-mat2-web-api-1 | spawned uWSGI master process (pid: 11)
test-mat2-web-api-1 | spawned uWSGI worker 1 (pid: 18, cores: 1)
```https://0xacab.org/jvoisin/mat2-web/-/issues/63Release a new version with mat 0.12.42022-06-08T05:04:00ZcasperRelease a new version with mat 0.12.4Can you please release a new version of this image with the latest version of mat2? Thanks!Can you please release a new version of this image with the latest version of mat2? Thanks!https://0xacab.org/jvoisin/mat2-web/-/issues/62Docker container is not able to restart2022-04-24T11:19:44ZshadowDocker container is not able to restartThe docker container of mat2-web seems unable to restart and dies with `bind(): Permission denied [core/socket.c line 230]`, when running docker stop && docker start or docker restart or the server on which it is running reboots.
I assu...The docker container of mat2-web seems unable to restart and dies with `bind(): Permission denied [core/socket.c line 230]`, when running docker stop && docker start or docker restart or the server on which it is running reboots.
I assume the uWSGI has a problem on container restart.
``` shell
[15:07:07] foobar:~ $ docker run -d -p8181:8080 --name mat2 --read-only --tmpfs /tmp --tmpfs /run/uwsgi --tmpfs=/app/upload --security-opt=no-new-privileges registry.0xacab.org/jvoisin/mat2-web
0b9aef0973621297d27269e09e7d1f8de16d8afc50b5d5526a0c38db618333be
[15:07:54] foobar:~ $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0b9aef097362 registry.0xacab.org/jvoisin/mat2-web "sh -c 'nginx; uwsgi…" 3 seconds ago Up 2 seconds 0.0.0.0:8181->8080/tcp, :::8181->8080/tcp mat2
[15:07:56] foobar:~ $ docker logs mat2
[uWSGI] getting INI configuration from /etc/uwsgi/apps-enabled/mat2-web.ini
*** Starting uWSGI 2.0.18-debian (64bit) on [Mon Mar 14 14:07:54 2022] ***
compiled with version: 8.2.0 on 10 February 2019 02:42:46
os: Linux-5.4.0-104-generic #118-Ubuntu SMP Wed Mar 2 19:02:41 UTC 2022
nodename: 0b9aef097362
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /var/www/mat2-web
detected binary path: /usr/bin/uwsgi-core
chdir() to /var/www/mat2-web/
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /run/uwsgi/uwsgi.sock fd 3
Python version: 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x56525cc0f020
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145840 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x56525cc0f020 pid: 11 (default app)
mountpoint already configured. skip.
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 11)
spawned uWSGI worker 1 (pid: 16, cores: 1)
[15:08:01] foobar:~ $ docker stop mat2
mat2
[15:08:19] foobar:~ $ docker start mat2
mat2
[15:08:26] foobar:~ $ docker logs mat2
[uWSGI] getting INI configuration from /etc/uwsgi/apps-enabled/mat2-web.ini
*** Starting uWSGI 2.0.18-debian (64bit) on [Mon Mar 14 14:07:54 2022] ***
compiled with version: 8.2.0 on 10 February 2019 02:42:46
os: Linux-5.4.0-104-generic #118-Ubuntu SMP Wed Mar 2 19:02:41 UTC 2022
nodename: 0b9aef097362
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /var/www/mat2-web
detected binary path: /usr/bin/uwsgi-core
chdir() to /var/www/mat2-web/
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /run/uwsgi/uwsgi.sock fd 3
Python version: 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x56525cc0f020
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145840 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x56525cc0f020 pid: 11 (default app)
mountpoint already configured. skip.
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 11)
spawned uWSGI worker 1 (pid: 16, cores: 1)
[uWSGI] getting INI configuration from /etc/uwsgi/apps-enabled/mat2-web.ini
*** Starting uWSGI 2.0.18-debian (64bit) on [Mon Mar 14 14:08:24 2022] ***
compiled with version: 8.2.0 on 10 February 2019 02:42:46
os: Linux-5.4.0-104-generic #118-Ubuntu SMP Wed Mar 2 19:02:41 UTC 2022
nodename: 0b9aef097362
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /var/www/mat2-web
detected binary path: /usr/bin/uwsgi-core
chdir() to /var/www/mat2-web/
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
bind(): Permission denied [core/socket.c line 230]
```
A "workaround" is to remove the container and re-run it.
``` shell
[15:16:26] foobar:~ $ docker rm mat2
mat2
[15:16:32] foobar:~ $ docker run -d -p8181:8080 --name mat2 --read-only --tmpfs /tmp --tmpfs /run/uwsgi --tmpfs=/app/upload --security-opt=no-new-privileges registry.0xacab.org/jvoisin/mat2-web
60aefb0e906f7ccb75059ce768238a37cef6ea606348d40e26d68d624871924e
[15:16:41] foobar:~ $ docker logs mat2
[uWSGI] getting INI configuration from /etc/uwsgi/apps-enabled/mat2-web.ini
*** Starting uWSGI 2.0.18-debian (64bit) on [Mon Mar 14 14:16:38 2022] ***
compiled with version: 8.2.0 on 10 February 2019 02:42:46
os: Linux-5.4.0-104-generic #118-Ubuntu SMP Wed Mar 2 19:02:41 UTC 2022
nodename: 60aefb0e906f
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /var/www/mat2-web
detected binary path: /usr/bin/uwsgi-core
chdir() to /var/www/mat2-web/
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /run/uwsgi/uwsgi.sock fd 3
Python version: 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x55f9452c1020
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145840 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x55f9452c1020 pid: 12 (default app)
mountpoint already configured. skip.
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 12)
spawned uWSGI worker 1 (pid: 16, cores: 1)
```
Thank you for your software and also for your help!https://0xacab.org/jvoisin/mat2-web/-/issues/61Action Required: Fix Renovate Configuration2022-01-24T20:10:31ZjfriedliAction Required: Fix Renovate ConfigurationThere is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop MRs until it is resolved.
Location: `None`
Error type: An existing branch is blocking Renovate
Message: `Renovat...There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop MRs until it is resolved.
Location: `None`
Error type: An existing branch is blocking Renovate
Message: `Renovate needs to create the branch "renovate/flask-2.x" but is blocked from doing so because of an existing branch called "renovate". Please remove it so that Renovate can proceed.`https://0xacab.org/jvoisin/mat2-web/-/issues/60broken renovate CI job2022-01-24T20:55:38Zjfriedlibroken renovate CI jobhttps://0xacab.org/jvoisin/mat2-web/-/issues/59"bwrap: capset failed: Operation not permitted" when cleaning large pptx files2022-01-24T19:54:22ZL P"bwrap: capset failed: Operation not permitted" when cleaning large pptx filesI'm experiencing some issues when I try to clean large .pptx files.
I get the following error:
```python
web_1 | bwrap: capset failed: Operation not permitted
web_1 | Traceback (most recent call last):
web_1 | File "/usr/local/li...I'm experiencing some issues when I try to clean large .pptx files.
I get the following error:
```python
web_1 | bwrap: capset failed: Operation not permitted
web_1 | Traceback (most recent call last):
web_1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/exiftool.py", line 29, in get_meta
web_1 | check=True, stdout=subprocess.PIPE).stdout
web_1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/bubblewrap.py", line 106, in run
web_1 | completed_process = subprocess.run(prefix_args + args, **kwargs)
web_1 | File "/usr/lib/python3.7/subprocess.py", line 487, in run
web_1 | output=stdout, stderr=stderr)
web_1 | subprocess.CalledProcessError: Command '['/usr/bin/bwrap', '--ro-bind', '/usr', '/usr', '--ro-bind', '/lib', '/lib', '--ro-bind', '/lib64', '/lib64', '--ro-bind', '/bin', '/bin', '--ro-bind', '/sbin', '/sbin', '--ro-bind', '/etc/alternatives', '/etc/alternatives', '--ro-bind', '/var/www/mat2-web', '/var/www/mat2-web', '--ro-bind', '/etc/ld.so.cache', '/etc/ld.so.cache', '--dev', '/dev', '--proc', '/proc', '--chdir', '/var/www/mat2-web', '--unshare-user-try', '--unshare-ipc', '--unshare-pid', '--unshare-net', '--unshare-uts', '--unshare-cgroup-try', '--new-session', '--cap-drop', 'all', '--ro-bind', '/tmp/tmpo25dkbhr/docProps/thumbnail.jpeg', '/tmp/tmpo25dkbhr/docProps/thumbnail.jpeg', '/usr/bin/exiftool', '-json', '/tmp/tmpo25dkbhr/docProps/thumbnail.jpeg']' returned non-zero exit status 1.
web_1 |
web_1 | During handling of the above exception, another exception occurred:
web_1 |
web_1 | Traceback (most recent call last):
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 2464, in __call__
web_1 | return self.wsgi_app(environ, start_response)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 2450, in wsgi_app
web_1 | response = self.handle_exception(e)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask_cors/extension.py", line 165, in wrapped_function
web_1 | return cors_after_request(app.make_response(f(*args, **kwargs)))
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask_restful/__init__.py", line 272, in error_router
web_1 | return original_handler(e)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1867, in handle_exception
web_1 | reraise(exc_type, exc_value, tb)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/_compat.py", line 38, in reraise
web_1 | raise value.with_traceback(tb)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 2447, in wsgi_app
web_1 | response = self.full_dispatch_request()
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1952, in full_dispatch_request
web_1 | rv = self.handle_user_exception(e)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask_cors/extension.py", line 165, in wrapped_function
web_1 | return cors_after_request(app.make_response(f(*args, **kwargs)))
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask_restful/__init__.py", line 272, in error_router
web_1 | return original_handler(e)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1821, in handle_user_exception
web_1 | reraise(exc_type, exc_value, tb)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/_compat.py", line 38, in reraise
web_1 | raise value.with_traceback(tb)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1950, in full_dispatch_request
web_1 | rv = self.dispatch_request()
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1936, in dispatch_request
web_1 | return self.view_functions[rule.endpoint](**req.view_args)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask_restful/__init__.py", line 468, in wrapper
web_1 | resp = resource(*args, **kwargs)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/views.py", line 89, in view
web_1 | return self.dispatch_request(*args, **kwargs)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask_restful/__init__.py", line 583, in dispatch_request
web_1 | resp = meth(*args, **kwargs)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flasgger/utils.py", line 248, in wrapper
web_1 | return function(*args, **kwargs)
web_1 | File "./matweb/rest_api.py", line 120, in post
web_1 | _, _, _, output_filename = utils.cleanup(parser, filepath, current_app.config['UPLOAD_FOLDER'])
web_1 | File "./matweb/utils.py", line 86, in cleanup
web_1 | meta_after = parser.get_meta()
web_1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/archive.py", line 146, in get_meta
web_1 | local_meta = {**local_meta, **member_parser.get_meta()}
web_1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/exiftool.py", line 35, in get_meta
web_1 | raise ValueError
web_1 | ValueError
```
Cleaning the same file with mat2 is working absolutely fine.
I've tried to disable all bubblewrap calls in mat2, but afterwards I'm getting the following error:
```python
web_1 | b'[{\n "SourceFile": "/tmp/tmphv8su1ih/ppt/media/image10.svg",\n "ExifToolVersion": 11.16,\n "FileName": "image10.svg",\n "Directory": "/tmp/tmphv8su1ih/ppt/media",\n "FileSize": "12 kB",\n "FileModifyDate": "2022:01:05 10:59:26+00:00",\n "FileAccessDate": "2022:01:05 10:59:26+00:00",\n "FileInodeChangeDate": "2022:01:05 10:59:26+00:00",\n "FilePermissions": "r--------",\n "Error": "File format error"\n}]\n'
web_1 | Traceback (most recent call last):
web_1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/exiftool.py", line 27, in get_meta
web_1 | check=True, stdout=subprocess.PIPE).stdout
web_1 | File "/usr/lib/python3.7/subprocess.py", line 487, in run
web_1 | output=stdout, stderr=stderr)
web_1 | subprocess.CalledProcessError: Command '['/usr/bin/exiftool', '-json', '/tmp/tmphv8su1ih/ppt/media/image10.svg']' returned non-zero exit status 1.
web_1 |
web_1 | During handling of the above exception, another exception occurred:
web_1 |
web_1 | Traceback (most recent call last):
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 2464, in __call__
web_1 | return self.wsgi_app(environ, start_response)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 2450, in wsgi_app
web_1 | response = self.handle_exception(e)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask_cors/extension.py", line 165, in wrapped_function
web_1 | return cors_after_request(app.make_response(f(*args, **kwargs)))
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask_restful/__init__.py", line 272, in error_router
web_1 | return original_handler(e)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1867, in handle_exception
web_1 | reraise(exc_type, exc_value, tb)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/_compat.py", line 38, in reraise
web_1 | raise value.with_traceback(tb)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 2447, in wsgi_app
web_1 | response = self.full_dispatch_request()
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1952, in full_dispatch_request
web_1 | rv = self.handle_user_exception(e)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask_cors/extension.py", line 165, in wrapped_function
web_1 | return cors_after_request(app.make_response(f(*args, **kwargs)))
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask_restful/__init__.py", line 272, in error_router
web_1 | return original_handler(e)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1821, in handle_user_exception
web_1 | reraise(exc_type, exc_value, tb)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/_compat.py", line 38, in reraise
web_1 | raise value.with_traceback(tb)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1950, in full_dispatch_request
web_1 | rv = self.dispatch_request()
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1936, in dispatch_request
web_1 | return self.view_functions[rule.endpoint](**req.view_args)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask_restful/__init__.py", line 468, in wrapper
web_1 | resp = resource(*args, **kwargs)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask/views.py", line 89, in view
web_1 | return self.dispatch_request(*args, **kwargs)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flask_restful/__init__.py", line 583, in dispatch_request
web_1 | resp = meth(*args, **kwargs)
web_1 | File "/usr/local/lib/python3.7/dist-packages/flasgger/utils.py", line 248, in wrapper
web_1 | return function(*args, **kwargs)
web_1 | File "./matweb/rest_api.py", line 120, in post
web_1 | _, _, _, output_filename = utils.cleanup(parser, filepath, current_app.config['UPLOAD_FOLDER'])
web_1 | File "./matweb/utils.py", line 86, in cleanup
web_1 | meta_after = parser.get_meta()
web_1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/archive.py", line 146, in get_meta
web_1 | local_meta = {**local_meta, **member_parser.get_meta()}
web_1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/images.py", line 40, in get_meta
web_1 | meta = super().get_meta()
web_1 | File "/usr/local/lib/python3.7/dist-packages/libmat2/exiftool.py", line 30, in get_meta
web_1 | raise ValueError
web_1 | ValueError
```
I'm not quite sure if this error is really produced by mat2-web. But since mat2 is working fine with this file, this seems like the correct place.
BTW: Is it possible that bubblewrap is missing in the production dockerfile? Before adding apt install bubblewrap mat2-web is giving me an error, that bwrap was not found.https://0xacab.org/jvoisin/mat2-web/-/issues/58It's impossible to restart the container2021-11-26T13:56:36ZcasperIt's impossible to restart the containerWhen stopping the container and starting it again the daemon will fail. `docker logs` produces the following errors:
```
[uWSGI] getting INI configuration from /etc/uwsgi/apps-enabled/mat2-web.ini
*** Starting uWSGI 2.0.18-debian (64bit)...When stopping the container and starting it again the daemon will fail. `docker logs` produces the following errors:
```
[uWSGI] getting INI configuration from /etc/uwsgi/apps-enabled/mat2-web.ini
*** Starting uWSGI 2.0.18-debian (64bit) on [Fri Nov 26 12:02:53 2021] ***
compiled with version: 8.2.0 on 10 February 2019 02:42:46
os: Linux-5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30)
nodename: docker1
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 2
current working directory: /var/www/mat2-web
detected binary path: /usr/bin/uwsgi-core
chdir() to /var/www/mat2-web/
your processes number limit is 31701
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
bind(): Permission denied [core/socket.c line 230]
```
This does not happen, when creating the container for the first time or with other containers on the same machine.
A successful creation looks like this:
```
[uWSGI] getting INI configuration from /etc/uwsgi/apps-enabled/mat2-web.ini
*** Starting uWSGI 2.0.18-debian (64bit) on [Fri Nov 26 12:28:40 2021] ***
compiled with version: 8.2.0 on 10 February 2019 02:42:46
os: Linux-5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30)
nodename: docker1
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 2
current working directory: /var/www/mat2-web
detected binary path: /usr/bin/uwsgi-core
chdir() to /var/www/mat2-web/
your processes number limit is 31701
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /run/uwsgi/uwsgi.sock fd 3
Python version: 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x555997e8c020
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145840 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x555997e8c020 pid: 13 (default app)
mountpoint already configured. skip.
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 13)
spawned uWSGI worker 1 (pid: 16, cores: 1)
```https://0xacab.org/jvoisin/mat2-web/-/issues/57Bug: Success download shield logo is not centered2021-11-01T15:52:32ZjfriedliBug: Success download shield logo is not centeredhttps://0xacab.org/jvoisin/mat2-web/-/issues/56/api/remove_matadata endpoint fails when multipart body is streamed2021-12-17T19:24:13ZDW/api/remove_matadata endpoint fails when multipart body is streamedWhen requesting to endpoint `/api/remove_metadata` without `Content-Length` HTTP header, the API returns 400 with message `"No file part"`, even though the body exists and is encoded correctly.
Request without `Content-Length` is a vali...When requesting to endpoint `/api/remove_metadata` without `Content-Length` HTTP header, the API returns 400 with message `"No file part"`, even though the body exists and is encoded correctly.
Request without `Content-Length` is a valid http request when using chunked transfer encoding ([Wikipedia](https://en.wikipedia.org/wiki/Chunked_transfer_encoding), [MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding)). Such method is useful when streaming file from network/disk directly without loading the entire file to memory (and thus the client making the api request not aware of the file size).
Reproducable in Docker (built with production Dockerfile).
Wireshark capture of failing request:
![image](/uploads/94d53ef1a7e1fc932981988793225d93/image.png)
Succeeding request:
![image](/uploads/e09ef295a2d7b0d1d2ad6bf7a9568a88/image.png)https://0xacab.org/jvoisin/mat2-web/-/issues/55Internal error when removing .xlsx metadata2021-03-23T20:34:07ZLukas ParkerInternal error when removing .xlsx metadataWhenever I try to remove .xlsx Metadata, I get the following error:
> web_1 | INFO:werkzeug:172.18.0.1 - - [08/Mar/2021 16:12:38] "POST /api/remove_metadata HTTP/1.1" 200 -
> web_1 | ERROR:root:In file ./uploads/Mappe1.xlsx, element x...Whenever I try to remove .xlsx Metadata, I get the following error:
> web_1 | INFO:werkzeug:172.18.0.1 - - [08/Mar/2021 16:12:38] "POST /api/remove_metadata HTTP/1.1" 200 -
> web_1 | ERROR:root:In file ./uploads/Mappe1.xlsx, element xl/_rels/workbook.xml.rels's format (None) isn't supported
> web_1 | ERROR:root:In file ./uploads/Mappe1.xlsx, element xl/styles.xml's format (application/xml) isn't supported
> web_1 | ERROR:root:In file ./uploads/Mappe1.xlsx, element xl/theme/theme1.xml's format (application/xml) isn't supported
> web_1 | ERROR:root:In file ./uploads/Mappe1.xlsx, element xl/workbook.xml's format (application/xml) isn't supported
> web_1 | ERROR:root:In file ./uploads/Mappe1.xlsx, element xl/worksheets/sheet1.xml's format (application/xml) isn't supported
> web_1 | [2021-03-08 16:13:27,879] ERROR in rest_api: Clean - Unable to clean application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
> web_1 | ERROR:main:Clean - Unable to clean application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
> web_1 | [2021-03-08 16:13:27,880] ERROR in app: Exception on /api/remove_metadata [POST]
> web_1 | Traceback (most recent call last):
> web_1 | File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request
> web_1 | rv = self.dispatch_request()
> web_1 | File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
> web_1 | return self.view_functions[rule.endpoint](**req.view_args)
> web_1 | File "/usr/local/lib/python3.9/site-packages/flask_restful/__init__.py", line 468, in wrapper
> web_1 | resp = resource(*args, **kwargs)
> web_1 | File "/usr/local/lib/python3.9/site-packages/flask/views.py", line 89, in view
> web_1 | return self.dispatch_request(*args, **kwargs)
> web_1 | File "/usr/local/lib/python3.9/site-packages/flask_restful/__init__.py", line 583, in dispatch_request
> web_1 | resp = meth(*args, **kwargs)
> web_1 | File "/usr/local/lib/python3.9/site-packages/flasgger/utils.py", line 248, in wrapper
> web_1 | return function(*args, **kwargs)
> web_1 | File "/mat2-web/matweb/rest_api.py", line 122, in post
> web_1 | abort(500, message='Unable to clean %s' % mime)
> web_1 | File "/usr/local/lib/python3.9/site-packages/flask_restful/__init__.py", line 32, in abort
> web_1 | original_flask_abort(http_status_code)
> web_1 | File "/usr/local/lib/python3.9/site-packages/werkzeug/exceptions.py", line 822, in abort
> web_1 | return _aborter(status, *args, **kwargs)
> web_1 | File "/usr/local/lib/python3.9/site-packages/werkzeug/exceptions.py", line 807, in __call__
> web_1 | raise self.mapping[code](*args, **kwargs)
> web_1 | werkzeug.exceptions.InternalServerError: 500 Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
I also tried if https://matweb.info/ is able to remove .xlsx metadata, but I also get an error there.https://0xacab.org/jvoisin/mat2-web/-/issues/54master pipeline failing2021-01-29T10:43:27Zjfriedlimaster pipeline failingSomehow the tests started to fail on master.
The extension .asc is not in the supported extensions list anymore.Somehow the tests started to fail on master.
The extension .asc is not in the supported extensions list anymore.https://0xacab.org/jvoisin/mat2-web/-/issues/53Run on subpath2020-10-14T14:39:29ZngRun on subpathI'd like to run the app/container on a subpath (e.g. `/mat2web`), though nginx but also flask seems to always assume to run on `/`. So script and assets path are always starting with `/`.
I do not yet fully understand how flask & uwsgi ...I'd like to run the app/container on a subpath (e.g. `/mat2web`), though nginx but also flask seems to always assume to run on `/`. So script and assets path are always starting with `/`.
I do not yet fully understand how flask & uwsgi plays together with regards to have it run on a subpath and whether any code change is actually needed (except for config changes), BUT it likely requires some adaption of the nginx templates on the fly.
My proposal would be to have the entrypoint checking on a ENV variable (e.g. MAT2WEB_SUBPATH) and if present, rewrite the templates (maybe nginx can even do it dynamically) and make sure flask is aware of it (e.g. by setting SCRIPT_NAME).
Motiviation: This would allow to bundle mat2web with other webapps, without having to host in on a separate subdomain.https://0xacab.org/jvoisin/mat2-web/-/issues/52Container fails to start on systems without IPv6 (or IPv4)2020-10-14T14:39:31ZngContainer fails to start on systems without IPv6 (or IPv4)Nginx is hardcoded to listen on IPv6 loopback: https://0xacab.org/jvoisin/mat2-web/-/blob/master/config/nginx-default.conf#L4
If you run this container on a system without IPv6 (disabled via kernel cmdline), you get the following:
```b...Nginx is hardcoded to listen on IPv6 loopback: https://0xacab.org/jvoisin/mat2-web/-/blob/master/config/nginx-default.conf#L4
If you run this container on a system without IPv6 (disabled via kernel cmdline), you get the following:
```bash
$ podman run -ti -p8181:8080 --read-only --tmpfs /tmp --tmpfs /run/uwsgi --tmpfs=/app/upload --security-opt=no-new-privileges registry.0xacab.org/jvoisin/mat2-web:latest
2020/09/22 12:42:46 [emerg] 8#8: socket() [::]:8080 failed (97: Address family not supported by protocol)
nginx: [emerg] socket() [::]:8080 failed (97: Address family not supported by protocol)
[uWSGI] getting INI configuration from /etc/uwsgi/apps-enabled/mat2-web.ini
[...]
```
Though the container is running fine, as uwsgi runs and thus the main process does not exit...
Likely you get the same issue on systems without IPv4, though not sure how Line 3 of the nginx config behaves, when no IPv4 stack is present.
Maybe just listen on Line 3 (without `[::]` on Line 4) would be sufficient?
Mainly recording the error here in case someone else tries to to debug the same issue.https://0xacab.org/jvoisin/mat2-web/-/issues/51ssr2020-08-16T12:35:17Zjfriedlissrhttps://quasar.dev/quasar-cli/developing-ssr/introduction
https://quasar.dev/quasar-cli/prefetch-feature
https://medium.com/@ramansah/server-side-rendered-open-graph-meta-tags-in-vue-js-31d864004306https://quasar.dev/quasar-cli/developing-ssr/introduction
https://quasar.dev/quasar-cli/prefetch-feature
https://medium.com/@ramansah/server-side-rendered-open-graph-meta-tags-in-vue-js-31d864004306https://0xacab.org/jvoisin/mat2-web/-/issues/50Translation files2020-08-05T18:25:06ZemmapeelTranslation filesIt would be great to generate some .po files so we can translate MAT2 to other languages.
The sooner the better, as translators will also need some time to translate...It would be great to generate some .po files so we can translate MAT2 to other languages.
The sooner the better, as translators will also need some time to translate...https://0xacab.org/jvoisin/mat2-web/-/issues/49bulk endpoint throws 500 on empty body2020-07-14T20:59:59Zjfriedlibulk endpoint throws 500 on empty body