Merge pull request #1 from geopozo/andrew/api_refactor

Andrew/api refactor

Author: ayjayt

.github/workflows/ruff.yml

@@ -7,5 +7,3 @@ jobs:
     steps:
       - uses: actions/checkout@v4
       - uses: chartboost/ruff-action@v1
-        with:
-          src: './logistro'

.pre-commit-config.yaml

@@ -0,0 +1,90 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+%YAML 1.2
+---
+exclude: 'site/.*'
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v6.0.0
+    hooks:
+      - id: trailing-whitespace
+      - id: end-of-file-fixer
+      - id: check-yaml
+      - id: check-added-large-files
+      - id: check-case-conflict
+      - id: check-merge-conflict
+      - id: check-toml
+      - id: debug-statements
+  - repo: https://github.com/asottile/add-trailing-comma
+    rev: v4.0.0
+    hooks:
+      - id: add-trailing-comma
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    #  Ruff version.
+    rev: v0.14.3
+    hooks:
+      #  Run the linter.
+      - id: ruff
+        types_or: [python, pyi]
+      #  Run the formatter.
+      - id: ruff-format
+        types_or: [python, pyi]
+        #  options: ignore one line things [E701]
+  - repo: https://github.com/adrienverge/yamllint
+    rev: v1.37.1
+    hooks:
+      - id: yamllint
+        name: yamllint
+        description: This hook runs yamllint.
+        entry: yamllint
+        language: python
+        types: [file, yaml]
+        args: [
+          '-d',
+          "{ extends: default, rules: { colons: { max-spaces-after: -1 } } }",
+        ]
+  - repo: https://github.com/rhysd/actionlint
+    rev: v1.7.8
+    hooks:
+      - id: actionlint
+        name: Lint GitHub Actions workflow files
+        description: Runs actionlint to lint GitHub Actions workflow files
+        language: golang
+        types: ["yaml"]
+        files: ^\.github/workflows/
+        entry: actionlint
+  - repo: https://github.com/jorisroovers/gitlint
+    rev: v0.19.1
+    hooks:
+      - id: gitlint
+        name: gitlint
+        description: Checks your git commit messages for style.
+        language: python
+        additional_dependencies: ["./gitlint-core[trusted-deps]"]
+        entry: gitlint
+        args: [--staged, --msg-filename]
+        stages: [commit-msg]
+  - repo: https://github.com/crate-ci/typos
+    rev: v1
+    hooks:
+      - id: typos
+  - repo: https://github.com/Yelp/detect-secrets
+    rev: v1.5.0
+    hooks:
+      - id: detect-secrets
+        name: Detect secrets
+        language: python
+        entry: detect-secrets-hook
+        args: ['']
+  - repo: https://github.com/rvben/rumdl-pre-commit
+    rev: v0.0.170  # Use the latest release tag
+    hooks:
+      - id: rumdl
+        # To only check (default):
+        # args: []
+        # To automatically fix issues:
+        # args: [--fix]
+  - repo: https://github.com/RobertCraigie/pyright-python
+    rev: v1.1.407   # pin a tag; latest as of 2025-10-01
+    hooks:
+      - id: pyright

CHANGELOG.txt

@@ -0,0 +1,2 @@
+v0.2.0
+- Refactor to improve internal/external API

README.md

@@ -1,26 +1,112 @@
-Later todo
-  - [ ] Add checking if git dirty or not pushed
-  - [ ] Add some log statistics
-  - [ ] Add some firewall statistics
-  - [ ] Add updates
-  - [ ] Add changelog (linux)
-  - [ ] Add news, linux, hackernews?
-  - [ ] Add website status check
-  - [ ] Add google analytics check
-  - [ ] But also constant ingress and event push (tetsuya)
-  - [ ] Add something that checks on any request
-  - [ ] Check itself
-
-
-# Get it up on systemd
+# tetsuya
 
-```
+tetsuya collects information and offers it up as JSON packets and pretty
+print strings.
+
+It's *very* easy to extend:
+
+- Write a function (a *service*) that collects information from web/system.
+  - It takes a `dataclass` as a *config* object (can be empty).
+  - It returns another `dataclass` as a *report*.
+
+tetsuya offers the *report* object to your user (and your user only) as a JSON
+endpoint available via REST and command-line interface.
+
+You will also define two functions on your *report* dataclass: `short()` and
+`long()` which can pretty-print the JSON.
+
+It does some helpful stuff automatically:
+
+1. tetsuya can cache your new *service* for you and auto-refresh the cache: all
+   tetsuya service configs have a cachelife integer and autorefresh boolean.
+
+2. tetsuya will derive the default config from your `dataclass`, but will read a
+   .toml if you want to change it.
+
+3. tetsuya runs in a client-server model, with a background daemon doing the
+   work, and a CLI interface for basic control.
+
+Try `uvx tetsuya --help-tree=ascii` to see the whole interface.
+
+## Installation
+
+### Daemon on systemd
+
+Theres a *tetsuya.service* file in the repository.
+
+```bash
 mkdir -p ~/.config/systemd/user
 
 systemctl --user daemon-reload
-systemctl --user enable --now "$(realpath tetsuya.service)"
+systemctl --user enable --now "$(realpath ./tetsuya.service)"
 
-loginctl enable-linger "$USER" # (allow it to start at boot)
+loginctl enable-linger "$USER" # (allow it to start at boot even if logged out)
 
 journalctl --user -u tetsuya -f
 ```
+
+## Roadmap
+
+- [ ] Some early modules:
+  - [ ] Do a Basic 200 is it good thing
+    - [ ] Check domains for email record
+    - [ ] Check SLL
+  - [ ] Active sessions on linux
+  - [ ] Updates available
+    - [ ] Changelog on kernel
+
+    ```bash
+    if which yay 1> /dev/null; then
+      (
+      set -e
+      yay -Qu
+      yay -Pw
+      checkupdates
+      )
+    fi
+    ```
+
+- [ ] Any errors on systemd + --kernel
+- [ ] Monarch Money from that guy? - MoneyFlown and redeploy
+- [ ] Improve naming and arguments, flags, etc.
+  - [ ] -> Services to Client
+  - [ ] Names of services = arguments
+  - [ ] Add help descriptions
+  - [ ] Turn off options like --thing, --no-thing, if --no-thing is default?
+  - [ ] Go back to CLI and do the formatting better
+- [ ] Document throughout code
+- [ ] If we want to instantiate multiple instances of one service class:
+- [ ]  Inspect:
+  - [ ] all instances of get_name()
+  - [ ] how services are registered
+  - [ ] Consider Dictionary Storage and Access Here:
+    - [ ] `_config.config_data`
+    - [ ] `_timer.timer_tasks`
+    - [ ] `service manager.??` (Not written at this time)
+  - [ ] Will have to pass both service obj + class to _config, _timer
+  - [ ] Make room for customed name in config + `.get_name()`
+  - [ ] Start will have change (calls a lot of this stuff)
+- [ ] Don't enable service until it has a config
+  - [ ] Allow per app config default generation
+  - [ ] Upon reload, recalculate active services
+  - [ ] Config API:
+    - [ ] Separate Touch and Dump
+- [ ] Create persistence:
+  - [ ] Roundtrip reports upon registering (json and back)
+  - [ ] Save reports upon running
+  - [ ] Load reports upong starting
+- [ ] Services can subscribe to changes of other services
+- [ ] Importing modules dynamically from a subfoler
+
+### Desired Modules
+
+- [ ] Google drive auditor, google accounts/emails
+- [ ] Check agreements (modules subscribed to other modules)
+  - [ ] are we checking all domains that namecheap lists
+  - [ ] does google list all
+- [ ] Analytics summary + link
+
+- [ ] Git status
+- [ ] Scrape forum posts
+- [ ] Firewall stats
+- [ ] process accounting?

pyproject.toml

@@ -26,6 +26,7 @@ maintainers = [
 ]
 
 dependencies = [
+  #    "cli-tree[typer]",
     "fastapi>=0.118.0",
     "httpx>=0.28.1",
     "logistro>=1.1.0",
@@ -37,10 +38,13 @@ dependencies = [
     "uvicorn>=0.37.0",
 ]
 
+[tool.uv.sources]
+#cli-tree = { path = "../../utilities/cli-tree.git", editable = true }
+
 [project.urls]
 
 [project.scripts]
-tetsuya = "tetsuya.app:main"
+tetsuya = "tetsuya.service_manager:start_client"
 
 [dependency-groups]
 dev = [
@@ -106,3 +110,7 @@ help = "Run test by test, slowly, quitting after first error"
 [tool.poe.tasks.filter-test]
 cmd = "pytest --log-level=1 -W error -vvvx -rA --capture=no --show-capture=no"
 help = "Run any/all tests one by one with basic settings: can include filename and -k filters"
+
+[tool.pyright]
+venvPath = "."
+venv = ".venv"

src/tetsuya/__init__.py

@@ -1 +1 @@
-"""Provides a server and client app for checking in on user-space services."""
+"""Tetsuya collects information turns it into REST/cli endpoints."""

src/tetsuya/app/services/utils/cache.py src/tetsuya/_cache.pysrc/tetsuya/app/services/utils/cache.py renamed to src/tetsuya/_cache.py

@@ -0,0 +1,55 @@
+"""Tools for managing the global config."""
+
+from __future__ import annotations
+
+import tomllib
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+import logistro
+import platformdirs
+import tomli_w
+
+if TYPE_CHECKING:
+    from typing import Any
+
+    from .services._base import Han, Settei
+
+_logger = logistro.getLogger(__name__)
+
+config_file = (
+    Path(platformdirs.user_config_dir("tetsuya", "pikulgroup")) / "config.toml"
+)
+
+config_data: dict[Any, Any] = {}
+
+
+def load_config() -> bool:
+    if config_file.is_file():
+        with config_file.open("rb") as f:
+            config_data.clear()
+            config_data.update(tomllib.load(f))
+        return True
+    else:
+        _logger.info("No config file found.")
+        return False
+
+
+def get_active_config(service_def: Han) -> Settei | None:
+    # could cache
+    han = service_def
+    _d = config_data.get(han.service.get_name())
+    return han.config(**_d) if _d else None
+
+
+def set_default_config(service_def: Han, *, overwrite: bool = False) -> bool:
+    key = service_def.service.get_name()
+    if key in config_data and not overwrite:
+        return False
+    config_data[key] = service_def.config.default_config()
+    return True
+
+
+def write_config():
+    with config_file.open("wb") as f:
+        tomli_w.dump(config_data, f)

src/tetsuya/_config.py

@@ -0,0 +1,67 @@
+"""Server implements logic for starting and verifying server."""
+
+from __future__ import annotations
+
+import os
+import sys
+from http import HTTPStatus
+from typing import TYPE_CHECKING
+
+import httpx
+import logistro
+import uvicorn
+
+from tetsuya.core import daemon
+
+from .core.utils import get_http_client, uds_path
+
+if TYPE_CHECKING:
+    from pathlib import Path
+
+_logger = logistro.getLogger(__name__)
+
+
+def is_server_alive(uds_path: Path) -> bool:
+    """Check if server is running."""
+    if not uds_path.exists():
+        return False
+    client: None | httpx.Client = None
+    try:
+        client = get_http_client(uds_path, defer_close=False)
+        r = client.get("/ping")
+        if r.status_code == HTTPStatus.OK:
+            _logger.info("Socket ping returned OK- server alive.")
+            return True
+        else:
+            _logger.info(
+                f"Socket ping returned {r.status_code}, removing socket.",
+            )
+            uds_path.unlink()
+            return False
+    except httpx.TransportError:
+        _logger.info("Transport error in socket, removing socket.")
+        uds_path.unlink()
+        return False
+    finally:
+        if client:
+            client.close()
+
+
+async def start():
+    if not is_server_alive(p := uds_path()):
+        os.umask(0o077)
+        _logger.info("Starting server.")
+        server = uvicorn.Server(
+            uvicorn.Config(
+                daemon,
+                uds=str(p),
+                loop="asyncio",
+                lifespan="on",
+                reload=False,
+            ),
+        )
+
+        await server.serve()  # calling the shortcut run would actually block the thread
+    else:
+        print("Server already running.", file=sys.stderr)  # noqa: T201
+        sys.exit(1)