[go: up one dir, main page]
Skip to content
This repository has been archived by the owner on May 27, 2022. It is now read-only.

fix: prevent OSError: read-only file system error #82

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

fix: prevent OSError: read-only file system error #82

wants to merge 1 commit into from

Conversation

daun-io
Copy link
@daun-io daun-io commented Mar 23, 2022 •
edited
Loading

Description

I found that there is a chance of OSError to occur when we try to load models into a temporary directory such as in the strictly managed environment like some containers on the cloud.

[2022-03-23 04:07:37,080] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]     review_scoring_model = Pororo(task="review", lang="ko")
[2022-03-23 04:07:37,080] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]   File "/usr/local/lib/python3.8/site-packages/pororo/pororo.py", line 203, in __new__
[2022-03-23 04:07:37,080] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]     task_module = SUPPORTED_TASKS[task](
[2022-03-23 04:07:37,080] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]   File "/usr/local/lib/python3.8/site-packages/pororo/tasks/review_scoring.py", line 86, in load
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]     model = (BrainRobertaModel.load_model(
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]   File "/usr/local/lib/python3.8/site-packages/pororo/models/brainbert/BrainRoBERTa.py", line 33, in load_model
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]     ckpt_dir = download_or_load(model_name, lang)
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]   File "/usr/local/lib/python3.8/site-packages/pororo/tasks/utils/download_utils.py", line 318, in download_or_load
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]     return download_or_load_bert(info)
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]   File "/usr/local/lib/python3.8/site-packages/pororo/tasks/utils/download_utils.py", line 104, in download_or_load_bert
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]     type_dir = download_from_url(
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]   File "/usr/local/lib/python3.8/site-packages/pororo/tasks/utils/download_utils.py", line 288, in download_from_url
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]     wget.download(url, type_dir)
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]   File "/usr/local/lib/python3.8/site-packages/wget.py", line 506, in download
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]     (fd, tmpfile) = tempfile.mkstemp(".tmp", prefix=prefix, dir=".")
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]   File "/usr/local/lib/python3.8/tempfile.py", line 331, in mkstemp
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]     return _mkstemp_inner(dir, prefix, suffix, flags, output_type)
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]   File "/usr/local/lib/python3.8/tempfile.py", line 250, in _mkstemp_inner
[2022-03-23 04:07:37,081] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000]     fd = _os.open(file, flags, 0o600)
[2022-03-23 04:07:37,082] {ecs.py:362} INFO - [2022-03-23T04:07:12.901000] OSError: [Errno 30] Read-only file system: './brainbert.base.ko.review_rating.zip4zkvg88b.tmp'

This commit will prevent that to happen. The code for the new function 'download' is originated from wget library written by anatoly techtonik with slight revision done by me.

fix: prevent OSError: read-only file system
0a07d1c 
I found that there is a chance of OSError to occur when we try to load models into a temporary directory such as in the strictly managed environment such as managed containers on the cloud.

This commit will prevent that to happen. The code for the new function 'download' is originated from wget library written by anatoly techtonik with slight revision done by me.
@daun-io daun-io changed the title fix: prevent OSError: read-only file system fix: prevent OSError: read-only file system error Mar 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@daun-io