mirror of
https://github.com/User0332/rewards-farmer.git
synced 2026-09-16 01:31:36 +00:00
run in docker, and work through more than one account
Both taken from TheNetsky/Microsoft-Rewards-Script, which packages a container and handles several accounts. Approach only: that project is GPL-3.0 and this one MIT, so no code crosses over. **Accounts.** Rewards is per Microsoft account and the browser profile holds the sign-in, so an account here is a profile directory. REWARDS_ACCOUNTS takes a comma separated list and gives each its own directory under the configured one. They run in sequence, and a profile that will not start is reported and skipped rather than ending the run. Left unset, a run uses the single profile exactly as before. Names are validated rather than trusted: they become directory names, so "../escape" is refused instead of quietly writing outside data-dir. **Docker.** The image carries only what main.py actually reaches, selenium and numpy. pygetwindow, keyboard, matplotlib and pygame are used solely by the recording and visualisation scripts, and two of those are Windows-only, so none of them belong in a container. msedgedriver is pinned at build time to the Edge the image installed rather than to latest, which drifts from it between releases. QUERY_SOURCE defaults to trends in the image, so a container needs no Ollama account and no model download at all. That default turned out to require a fix. queries.py imported llm_utils at module scope, which imports ollama, so a trends-only install still needed the ollama package: exactly what running in a minimal image is good at exposing. The import is now made inside the llm branch, and the wordlist fallback reads nouns.txt directly rather than borrowing llm_utils.get_random_noun. REWARDS_HEADLESS drives the headless flags. The window size is set explicitly because the pointer code works in viewport coordinates and the default headless window is small enough to put cards out of reach, which is the MoveTargetOutOfBoundsException from #19. Verified on the host that move_to_element and human_like_click both work headless before relying on it. Verified in the built image: Edge 151.0.4129.107 with a driver of exactly the same build, the trends feed reachable from inside, Edge driven to bing.com and rewards.bing.com at 1920x1080, and REWARDS_ACCOUNTS producing separate profile directories with traversal refused.
This commit is contained in:
@@ -0,0 +1,15 @@
|
||||
data-dir/
|
||||
venv/
|
||||
.venv/
|
||||
__pycache__/
|
||||
**/__pycache__/
|
||||
*.pyc
|
||||
.git/
|
||||
.gitignore
|
||||
.vscode/
|
||||
*.log
|
||||
*.png
|
||||
*.jpg
|
||||
poetry.lock
|
||||
README.md
|
||||
docs/
|
||||
+54
@@ -0,0 +1,54 @@
|
||||
# Runs the bot without installing Edge, a driver or Python on the host.
|
||||
#
|
||||
# The image carries only what main.py actually reaches: selenium and numpy.
|
||||
# pygetwindow, keyboard, matplotlib and pygame are used solely by the
|
||||
# recording and visualisation scripts, which are developer tools rather than
|
||||
# part of a run, and two of them are Windows-only.
|
||||
#
|
||||
# QUERY_SOURCE defaults to trends here so a container needs no Ollama account
|
||||
# and no model download. Set it to llm and point OLLAMA_HOST at a reachable
|
||||
# host to use a model instead.
|
||||
|
||||
FROM python:3.12-slim-bookworm
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
# Edge, from Microsoft's own repository.
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
ca-certificates curl gnupg unzip fonts-liberation \
|
||||
&& curl -fsSL https://packages.microsoft.com/keys/microsoft.asc \
|
||||
| gpg --dearmor -o /usr/share/keyrings/microsoft.gpg \
|
||||
&& echo "deb [arch=amd64 signed-by=/usr/share/keyrings/microsoft.gpg] https://packages.microsoft.com/repos/edge stable main" \
|
||||
> /etc/apt/sources.list.d/microsoft-edge.list \
|
||||
&& apt-get update \
|
||||
&& apt-get install -y --no-install-recommends microsoft-edge-stable \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# The driver has to match the browser build, so it is pinned to whatever Edge
|
||||
# the layer above installed rather than to "latest", which drifts apart from it
|
||||
# between releases.
|
||||
RUN EDGE_VERSION="$(microsoft-edge --version | awk '{print $3}')" \
|
||||
&& curl -fsSL -o /tmp/edgedriver.zip \
|
||||
"https://msedgedriver.microsoft.com/${EDGE_VERSION}/edgedriver_linux64.zip" \
|
||||
&& unzip -j /tmp/edgedriver.zip msedgedriver -d /usr/local/bin \
|
||||
&& chmod +x /usr/local/bin/msedgedriver \
|
||||
&& rm /tmp/edgedriver.zip \
|
||||
&& msedgedriver --version
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN pip install --no-cache-dir "selenium>=4.46.0,<5.0.0" "numpy"
|
||||
|
||||
COPY src/ ./src/
|
||||
COPY nouns.txt ./
|
||||
|
||||
# Headless because there is no display, and trends because there is no model.
|
||||
ENV REWARDS_HEADLESS=1 \
|
||||
QUERY_SOURCE=trends \
|
||||
PYTHONUNBUFFERED=1
|
||||
|
||||
# Sign-in lives here, so it has to outlive the container.
|
||||
VOLUME ["/app/data-dir"]
|
||||
|
||||
CMD ["python", "src/main.py"]
|
||||
@@ -66,4 +66,47 @@ EU Users: you may have to accept a consent banner once on `rewards.bing.com` and
|
||||
|
||||
Close all webdriver browser instances. Run `main.py` again; the automation should start working.
|
||||
|
||||
# Running more than one account
|
||||
|
||||
Rewards is per Microsoft account and the browser profile holds the sign-in, so an account here is a profile directory. `REWARDS_ACCOUNTS` takes a comma separated list, and each name gets its own directory under `data-dir`:
|
||||
|
||||
```sh
|
||||
REWARDS_ACCOUNTS=personal,spare python src/main.py
|
||||
```
|
||||
|
||||
Each is signed in once by hand, the same way as the single profile, using its own directory:
|
||||
|
||||
```
|
||||
msedge --user-data-dir="<repo>\data-dir\personal" --profile-directory=Default https://rewards.bing.com
|
||||
```
|
||||
|
||||
They run one after another, and a profile that fails to start is reported and skipped rather than ending the run. Leave `REWARDS_ACCOUNTS` unset and everything behaves exactly as before, using the single profile in `data-dir`.
|
||||
|
||||
# Docker
|
||||
|
||||
Runs the bot without installing Edge, a driver or Python on the host.
|
||||
|
||||
```sh
|
||||
docker compose build
|
||||
docker compose run --rm rewards-farmer
|
||||
```
|
||||
|
||||
The container defaults to `QUERY_SOURCE=trends`, so it needs no Ollama account and no model. Set `QUERY_SOURCE=llm` and `OLLAMA_HOST` to a reachable address to use a model instead.
|
||||
|
||||
**Sign in first.** The profile in `data-dir` starts logged out and the container has no display to sign in with, so do it once on the host with a normal Edge window and let the volume carry it in:
|
||||
|
||||
```
|
||||
msedge --user-data-dir="<repo>\data-dir" --profile-directory=Default https://rewards.bing.com
|
||||
```
|
||||
|
||||
Close every window of that profile afterwards. Chromium allows one process per profile directory, so a window left open on the host stops the container from starting.
|
||||
|
||||
Multiple accounts work the same way in the container:
|
||||
|
||||
```sh
|
||||
REWARDS_ACCOUNTS=personal,spare docker compose run --rm rewards-farmer
|
||||
```
|
||||
|
||||
`REWARDS_HEADLESS=1` is set in the image. It also works on the host if you want a run with no visible window; the pointer code needs an explicit window size in that mode, which `main.py` sets.
|
||||
|
||||
Please open up a GitHub issue if you run into any difficulties.
|
||||
@@ -0,0 +1,29 @@
|
||||
# docker compose run --rm rewards-farmer
|
||||
#
|
||||
# Sign-in has to happen once by hand before this is useful: the profile in
|
||||
# data-dir starts logged out. See the Docker section of the README.
|
||||
|
||||
services:
|
||||
rewards-farmer:
|
||||
build: .
|
||||
image: rewards-farmer
|
||||
# Chromium wants more than the default 64MB of shared memory and crashes
|
||||
# partway through a page without it.
|
||||
shm_size: 1gb
|
||||
environment:
|
||||
# trends needs no account or model, which is why it is the default here.
|
||||
# To use a model instead, set QUERY_SOURCE=llm and give OLLAMA_HOST an
|
||||
# address the container can reach: localhost inside a container is the
|
||||
# container, so ollama running on the host is host.docker.internal, and
|
||||
# 0.0.0.0 is a bind address that cannot be dialled at all.
|
||||
#
|
||||
# QUERY_SOURCE=llm OLLAMA_HOST=host.docker.internal:11434 docker compose run --rm rewards-farmer
|
||||
QUERY_SOURCE: ${QUERY_SOURCE:-trends}
|
||||
REWARDS_HEADLESS: "1"
|
||||
# Comma separated, one profile directory each. Leave unset for a single
|
||||
# profile in data-dir, which is the existing behaviour.
|
||||
REWARDS_ACCOUNTS: ${REWARDS_ACCOUNTS:-}
|
||||
OLLAMA_HOST: ${OLLAMA_HOST:-}
|
||||
volumes:
|
||||
# Keeps the sign-in across container rebuilds.
|
||||
- ./data-dir:/app/data-dir
|
||||
@@ -0,0 +1,85 @@
|
||||
"""Which accounts a run works through.
|
||||
|
||||
Rewards is per Microsoft account, and the browser profile is what holds the
|
||||
sign-in, so an account here is just a profile directory. One directory per
|
||||
account keeps their cookies apart, which is the whole requirement.
|
||||
|
||||
REWARDS_ACCOUNTS=personal,spare python src/main.py
|
||||
|
||||
Unset, the run uses the single profile in constants.py exactly as before, so
|
||||
nothing about an existing setup changes.
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
|
||||
from constants import USER_DATA_DIR, PROFILE_NAME
|
||||
|
||||
ENV_VAR = "REWARDS_ACCOUNTS"
|
||||
|
||||
# Names become directory names, so keep them to something a filesystem and a
|
||||
# command line both handle without quoting.
|
||||
SAFE_NAME = re.compile(r"^[A-Za-z0-9._-]+$")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Account:
|
||||
"""A named browser profile to run the tasks against."""
|
||||
|
||||
name: str
|
||||
user_data_dir: str
|
||||
profile_name: str
|
||||
|
||||
@property
|
||||
def is_default(self) -> bool:
|
||||
return self.user_data_dir == USER_DATA_DIR
|
||||
|
||||
|
||||
def _named(name: str) -> Account:
|
||||
# Each account gets its own directory under the configured one, so the
|
||||
# existing data-dir stays where it is and the new ones sit beside the
|
||||
# profile it already holds.
|
||||
return Account(
|
||||
name=name,
|
||||
user_data_dir=os.path.join(USER_DATA_DIR, name),
|
||||
profile_name=PROFILE_NAME,
|
||||
)
|
||||
|
||||
|
||||
def configured() -> list[Account]:
|
||||
"""Accounts for this run, in order.
|
||||
|
||||
Raises ValueError on a name that cannot be a directory, rather than
|
||||
silently creating something surprising next to the real profiles.
|
||||
"""
|
||||
raw = os.environ.get(ENV_VAR, "").strip()
|
||||
|
||||
if not raw:
|
||||
return [Account(name="default", user_data_dir=USER_DATA_DIR, profile_name=PROFILE_NAME)]
|
||||
|
||||
names = [part.strip() for part in raw.split(",")]
|
||||
names = [name for name in names if name]
|
||||
|
||||
if not names:
|
||||
return [Account(name="default", user_data_dir=USER_DATA_DIR, profile_name=PROFILE_NAME)]
|
||||
|
||||
seen: set[str] = set()
|
||||
accounts: list[Account] = []
|
||||
|
||||
for name in names:
|
||||
if not SAFE_NAME.match(name):
|
||||
raise ValueError(
|
||||
f"{ENV_VAR} entry {name!r} is not usable as a directory name; "
|
||||
"use letters, digits, dot, dash or underscore"
|
||||
)
|
||||
|
||||
# Duplicates would run the same profile twice, which earns nothing the
|
||||
# second time and doubles the run length.
|
||||
if name.lower() in seen:
|
||||
continue
|
||||
|
||||
seen.add(name.lower())
|
||||
accounts.append(_named(name))
|
||||
|
||||
return accounts
|
||||
+72
-6
@@ -1,26 +1,92 @@
|
||||
import os
|
||||
import sys
|
||||
|
||||
import accounts
|
||||
import rewards_tasks
|
||||
import mouse_trajectory
|
||||
import mimic_typing
|
||||
from selenium import webdriver
|
||||
from constants import USER_DATA_DIR, PROFILE_NAME
|
||||
from selenium.common.exceptions import SessionNotCreatedException
|
||||
|
||||
HEADLESS = os.environ.get("REWARDS_HEADLESS", "").strip().lower() in ("1", "true", "yes")
|
||||
|
||||
|
||||
def build_options(account: accounts.Account) -> webdriver.EdgeOptions:
|
||||
options = webdriver.EdgeOptions()
|
||||
|
||||
options.add_experimental_option("excludeSwitches", ["enable-automation"])
|
||||
options.add_experimental_option('useAutomationExtension', False)
|
||||
options.add_argument("--disable-blink-features=AutomationControlled")
|
||||
options.add_argument(f"--user-data-dir={USER_DATA_DIR}")
|
||||
options.add_argument(f"--profile-directory={PROFILE_NAME}")
|
||||
options.add_argument(f"--user-data-dir={account.user_data_dir}")
|
||||
options.add_argument(f"--profile-directory={account.profile_name}")
|
||||
|
||||
driver = webdriver.Edge(options=options)
|
||||
if HEADLESS:
|
||||
# A container has no display. The window size is set explicitly because
|
||||
# the pointer code works in viewport coordinates, and the default
|
||||
# headless window is small enough to put cards out of reach.
|
||||
options.add_argument("--headless=new")
|
||||
options.add_argument("--window-size=1920,1080")
|
||||
options.add_argument("--no-sandbox")
|
||||
options.add_argument("--disable-dev-shm-usage")
|
||||
|
||||
return options
|
||||
|
||||
|
||||
def run_account(account: accounts.Account) -> bool:
|
||||
"""Work one account. Returns whether the browser started."""
|
||||
try:
|
||||
driver = webdriver.Edge(options=build_options(account))
|
||||
except SessionNotCreatedException as exc:
|
||||
# Chromium allows one process per user data directory. When the profile
|
||||
# is already open the driver's copy exits during startup, and selenium
|
||||
# reports it as the browser crashing with a message that names neither
|
||||
# the profile nor the other window.
|
||||
print(f"[FAIL] {account.name}: could not start Edge with this profile.")
|
||||
print(f" profile directory: {account.user_data_dir}")
|
||||
print(" The usual cause is that this profile is already open in another")
|
||||
print(" Edge window, including one left over from a previous run.")
|
||||
print(f" driver said: {str(exc).strip().splitlines()[0]}")
|
||||
|
||||
return False
|
||||
|
||||
try:
|
||||
mouse = mouse_trajectory.MouseUtils(driver)
|
||||
keyboard = mimic_typing.KeyboardUtils(driver)
|
||||
|
||||
rewards = rewards_tasks.RewardsTaskUtils(driver)
|
||||
|
||||
rewards.complete_all_tasks()
|
||||
finally:
|
||||
driver.quit()
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def main() -> int:
|
||||
try:
|
||||
configured = accounts.configured()
|
||||
except ValueError as exc:
|
||||
print(f"[FAIL] {exc}")
|
||||
|
||||
return 2
|
||||
|
||||
started = 0
|
||||
|
||||
for account in configured:
|
||||
if len(configured) > 1:
|
||||
print(f"\n=== account: {account.name} ===")
|
||||
|
||||
if run_account(account):
|
||||
started += 1
|
||||
|
||||
if len(configured) > 1:
|
||||
print(f"\n{started}/{len(configured)} accounts ran")
|
||||
|
||||
# Nothing is watching a container, and stdin is not a terminal there.
|
||||
if not HEADLESS:
|
||||
input("Press Enter to exit...")
|
||||
|
||||
driver.quit()
|
||||
return 0 if started else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
||||
+11
-2
@@ -13,9 +13,14 @@ Bing, and Bing's own autosuggest answers that question directly.
|
||||
|
||||
import os
|
||||
|
||||
import llm_utils
|
||||
import query_sources
|
||||
|
||||
# llm_utils is imported inside the llm branch rather than here. It imports
|
||||
# ollama at module scope, so importing it eagerly would make the ollama package
|
||||
# a hard requirement even for a run that never touches a model, which is the
|
||||
# opposite of the point. A trends-only install, the Docker image for instance,
|
||||
# does not ship it.
|
||||
|
||||
LLM = "llm"
|
||||
TRENDS = "trends"
|
||||
|
||||
@@ -45,6 +50,8 @@ def search_query_for_task(task_description: str) -> str:
|
||||
|
||||
return task_description.lower()
|
||||
|
||||
import llm_utils
|
||||
|
||||
return llm_utils.get_search_query_from_task_description(task_description)
|
||||
|
||||
|
||||
@@ -59,6 +66,8 @@ def related_queries(count: int):
|
||||
print("[WARNING] No query source reachable, falling back to the wordlist.")
|
||||
|
||||
# nouns.txt is already in the repo for exactly this kind of seed.
|
||||
return [llm_utils.get_random_noun() for _ in range(count)]
|
||||
return query_sources.wordlist_queries(count)
|
||||
|
||||
import llm_utils
|
||||
|
||||
return llm_utils.get_related_search_queries(llm_utils.get_random_noun(), num_queries=count)
|
||||
|
||||
@@ -130,6 +130,24 @@ def suggestions(seed: str) -> list[str]:
|
||||
return [_clean(s) for s in payload[1] if _clean(s)]
|
||||
|
||||
|
||||
def wordlist_queries(count: int) -> list[str]:
|
||||
"""Seeds from nouns.txt, the last resort when nothing is reachable.
|
||||
|
||||
Read here rather than borrowed from llm_utils so that a trends-only install
|
||||
never has to import the model client.
|
||||
"""
|
||||
try:
|
||||
with open("nouns.txt", encoding="utf-8") as handle:
|
||||
nouns = [line.strip().lower() for line in handle if len(line.strip()) >= 3]
|
||||
except OSError:
|
||||
return []
|
||||
|
||||
if not nouns:
|
||||
return []
|
||||
|
||||
return random.sample(nouns, min(count, len(nouns)))
|
||||
|
||||
|
||||
def _clean(text: str) -> str:
|
||||
"""Strip markup, collapse whitespace and drop punctuation Bing does not need."""
|
||||
text = re.sub(r"<[^>]+>", " ", text or "")
|
||||
|
||||
Reference in New Issue
Block a user