Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Andrew/refactor #178

Open
wants to merge 35 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
62f3564
Change subpackage name
ayjayt Jan 4, 2025
cc0898a
Remove system() shortcut
ayjayt Jan 5, 2025
140251c
Fix up _pipe.py for refactor
ayjayt Jan 5, 2025
46c58d6
Better organize _pipe for refactor:
ayjayt Jan 5, 2025
15f920f
Create a new browser package for brand support:
ayjayt Jan 5, 2025
c997d51
Fix formatting
ayjayt Jan 5, 2025
d8dbfec
Further fix up namespaces
ayjayt Jan 5, 2025
c2ebdf6
Refactor sys/cli
ayjayt Jan 6, 2025
d51fccc
Change names of protocol a bit
ayjayt Jan 6, 2025
241960d
Change namespace of protocol
ayjayt Jan 6, 2025
34d30c3
Add factored sys into chromium.py
ayjayt Jan 6, 2025
4e6eee4
Refactor wire out of pipe
ayjayt Jan 6, 2025
d7db1b5
Pull chrome constants out of which
ayjayt Jan 6, 2025
656cf71
Fix up namespaces and chromium.py
ayjayt Jan 6, 2025
73ff2e1
Add dev readmes
ayjayt Jan 6, 2025
3d30378
Create basic broker subpackage
ayjayt Jan 6, 2025
239e0d6
Init session after open
ayjayt Jan 6, 2025
4d8472b
Remove async version of browser before test
ayjayt Jan 6, 2025
5bde5fa
Refactor sync
ayjayt Jan 6, 2025
479c409
Fix imports
ayjayt Jan 6, 2025
3fbee4b
Fix up metadata
ayjayt Jan 6, 2025
1a1503f
Fix up namespaces
ayjayt Jan 6, 2025
7a3ae8a
Fix up import errors
ayjayt Jan 6, 2025
c4e79dc
Redo chromium.py changes?
ayjayt Jan 6, 2025
afaf7ad
Remember to return
ayjayt Jan 6, 2025
a592200
Move path to non-kwargs arg chromium
ayjayt Jan 6, 2025
7ae4fae
Fix some bad syntax
ayjayt Jan 6, 2025
3b68fc8
Fix syntax
ayjayt Jan 6, 2025
ac96c0b
Add process logger
ayjayt Jan 6, 2025
c7b2bf8
Improve logging
ayjayt Jan 6, 2025
c84ac06
Improve logging
ayjayt Jan 6, 2025
670fea2
Fix download path in CLI
ayjayt Jan 6, 2025
eef038c
Add lock to BrowserSync
ayjayt Jan 6, 2025
9f0d038
Make sure Chromium cleans its tmp dir
ayjayt Jan 7, 2025
2479441
Add logs and docs
ayjayt Jan 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/publish_testpypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
# don't modify sync file! messes up version!
- run: uv sync --all-extras --frozen # does order matter?
- run: uv build
- run: uv run --no-sync choreo_get_browser -v --i ${{ matrix.chrome_v }}
- run: uv run --no-sync choreo_get_chrome -v --i ${{ matrix.chrome_v }}
- name: Reinstall from wheel
run: >
uv pip install dist/choreographer-$(uv
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
- name: Install choreographer
run: uv sync --all-extras
- name: Install google-chrome-for-testing
run: uv run choreo_get_browser
run: uv run choreo_get_chrome
- name: Diagnose
run: uv run choreo_diagnose --no-run
timeout-minutes: 1
Expand Down
31 changes: 0 additions & 31 deletions choreographer/DIR_INDEX.txt

This file was deleted.

10 changes: 10 additions & 0 deletions choreographer/README.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
These READMEs (in directories) are development notes, think of them as additions
to the package-level docstrings (the one at the top of each __init__.py).


The browsers are the main entrypoint for the user. They:

1. Manage the actual browser process (start/stop/kill).
2. Integrate the different resources: (see _channels/, _brokers/, _browsers/).
3. Is a list of `Tab`s, `Session`s, and `Target`s.
4. Provides a basic API.
73 changes: 56 additions & 17 deletions choreographer/__init__.py
Original file line number Diff line number Diff line change
@@ -1,24 +1,63 @@
"""choreographer is a browser controller for python."""
"""
choreographer is a browser controller for python.

import choreographer._devtools_protocol_layer as protocol
choreographer is natively async, so while there are two main entrypoints:
classes `Browser` and `BrowserSync`, the sync version is very limited, functioning
as a building block.

from ._browser import Browser, BrowserClosedError, browser_which, get_browser_path
from ._cli_utils import get_browser, get_browser_sync
from ._pipe import BlockWarning, PipeClosedError
from ._system_utils._tempfile import TempDirectory, TempDirWarning
from ._tab import Tab
See the main README for a quickstart.
"""

__all__ = [
"BlockWarning",
"Browser",
from ._browsers import ( # noqa: F401 unused import
BrowserClosedError,
BrowserFailedError,
Chromium,
)
from ._channels import BlockWarning, ChannelClosedError # noqa: F401 unused import
from ._cli_utils import get_chrome, get_chrome_sync # noqa: F401 unused import
from ._sys_utils import ( # noqa: F401 unused import
TmpDirectory,
TmpDirWarning,
browser_which,
get_browser_path,
)
from .browser_sync import ( # noqa: F401 unused import
BrowserSync,
SessionSync,
TabSync,
TargetSync,
)

_sync_api = [
"BrowserSync",
"SessionSync",
"TabSync",
"TargetSync",
]

_browser_impls = [
"Chromium",
]

_errors = [
"BrowserClosedError",
"PipeClosedError",
"Tab",
"TempDirWarning",
"TempDirectory",
"BrowserFailedError",
"ChannelClosedError",
"BlockWarning",
"TmpDirWarning",
]

_utils = [
"get_chrome",
"get_chrome_sync",
"TmpDirectory",
"browser_which",
"get_browser",
"get_browser_path",
"get_browser_sync",
"protocol",
]

__all__ = [ # noqa: PLE0604 non-string in all
*_sync_api,
*_browser_impls,
*_errors,
*_utils,
]
13 changes: 13 additions & 0 deletions choreographer/_brokers/README.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Brokers will handle all events and promises, if there are any.

For example, the _sync.py broker only gives us the option of
starting a thread to dump all events. Writing a broker without
a concurrent structure would be fruitless- it would block and break.


The _async.py broker is much more complex, and will allow users to subscribe
to events, will fulfill processes when receiving responses from the browser
addressed to the user's commands, and generally absorbs all browser communication.

Brokers do not have a standard interface, they are usually 1-1 matched with a particular
`class Browser*` implementation.
5 changes: 5 additions & 0 deletions choreographer/_brokers/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from ._sync import BrokerSync

__all__ = [
"BrokerSync",
]
38 changes: 38 additions & 0 deletions choreographer/_brokers/_sync.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import json
from threading import Thread

import logistro

from choreographer import protocol
from choreographer._channels import ChannelClosedError

logger = logistro.getLogger(__name__)


class BrokerSync:
def __init__(self, browser, channel):
self.browser = browser
self.channel = channel

# This is the broker
def run_output_thread(self, **kwargs):
def run_print():
try:
while True:
responses = self.channel.read_jsons()
for response in responses:
print(json.dumps(response, indent=4), **kwargs)
except ChannelClosedError:
print("ChannelClosedError caught.", **kwargs)

logger.info("Starting thread to dump output to stdout.")
Thread(target=run_print).start()

def send_json(self, obj):
protocol.verify_params(obj)
key = protocol.calculate_message_key(obj)
self.channel.write_json(obj)
return key

def clean(self):
pass
Loading
Loading