mirror of
https://git.vern.cc/cobra/rural-dict.git
synced 2025-11-18 17:58:36 +05:30
RURAL DICTIONARY IS BACK
This commit is contained in:
Skylar "The Cobra" Astaroth
parent
70fd8f50d3
commit
f1bb994b1b
18 changed files with 194 additions and 580 deletions
|
|
@ -1,6 +0,0 @@
|
|||
*
|
||||
!requirements.lock
|
||||
!src/
|
||||
!templates/
|
||||
!static/
|
||||
static/**/*.xcf
|
||||
|
|
@ -1,13 +0,0 @@
|
|||
root = true
|
||||
|
||||
[**]
|
||||
charset = utf-8
|
||||
end_of_line = lf
|
||||
indent_size = 4
|
||||
indent_style = space
|
||||
insert_final_newline = true
|
||||
trim_trailing_whitespace = true
|
||||
max_line_length = 99
|
||||
|
||||
[{**.yml,**.yaml,**.html}]
|
||||
indent_size = 2
|
||||
|
|
@ -1 +0,0 @@
|
|||
3.12.4
|
||||
15
Dockerfile
15
Dockerfile
|
|
@ -1,7 +1,10 @@
|
|||
FROM python:3.12.4-alpine
|
||||
WORKDIR /app
|
||||
COPY requirements.lock ./
|
||||
RUN PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -r requirements.lock
|
||||
FROM python:3-alpine
|
||||
|
||||
WORKDIR /usr/src/rural-dict
|
||||
|
||||
COPY requirements.txt ./
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY . .
|
||||
CMD [ "uvicorn", "src.main:app", "--no-access-log", "--proxy-headers", \
|
||||
"--forwarded-allow-ips", "*", "--host", "0.0.0.0", "--port", "5758" ]
|
||||
|
||||
CMD [ "python", "./main.py" ]
|
||||
|
|
|
|||
122
README.md
122
README.md
|
|
@ -1,10 +1,8 @@
|
|||
# 📖 Rural Dictionary
|
||||
# Rural Dictionary
|
||||
|
||||
> We're rural, not urban.
|
||||
A privacy respecting JS-less Urban Dictionary client, powered by Flask.
|
||||
|
||||
Privacy-respecting, NoJS-supporting Urban Dictionary frontend.
|
||||
|
||||
## 🌐 Instances
|
||||
# Instances
|
||||
|
||||
| URL | Country | Owner name | Owner Website |
|
||||
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|-----------------|-------------------------------|
|
||||
|
|
@ -14,101 +12,47 @@ Privacy-respecting, NoJS-supporting Urban Dictionary frontend.
|
|||
| <https://rd.cleberg.net> | US | cleberg.net | <https://cleberg.net> |
|
||||
| <https://ruraldictionary.franklyflawless.org> | DE | FranklyFlawless | <https://franklyflawless.org> |
|
||||
|
||||
## ✨ Features
|
||||
# Support
|
||||
Join our [[https://mto.vern.cc/#/#cobra-frontends:vern.cc][Matrix room]] for support and other things related to Rural Dictionary
|
||||
|
||||
Frontend supports all Urban Dictionary features and has endpoint-parity with it.
|
||||
Available features include:
|
||||
|
||||
- Word definitions
|
||||
- Author pages
|
||||
- Homepage with words of the day
|
||||
- Random word definitions
|
||||
- 404 page with words similar to search
|
||||
- Pagination
|
||||
# Features
|
||||
- Define a word with multiple entries
|
||||
- Random list of words
|
||||
- User pages
|
||||
- Urban Dictionary home with words of the day
|
||||
- Matches urban dictionary's endpoints for features listed above
|
||||
|
||||
## 🚀 Deployment
|
||||
|
||||
Clone repository:
|
||||
|
||||
```sh
|
||||
git clone https://git.vern.cc/cobra/rural-dict.git
|
||||
cd rural-dict
|
||||
# Deployment
|
||||
Set up and activate a virtual environment with `venv`
|
||||
```
|
||||
python3 -m venv venv
|
||||
. venv/bin/activate
|
||||
```
|
||||
|
||||
### 🐳 With Docker
|
||||
|
||||
```sh
|
||||
docker build . -t rural-dict
|
||||
docker compose up -d
|
||||
Install dependencies
|
||||
```
|
||||
pip3 install -r requirements.txt
|
||||
```
|
||||
|
||||
### 💻 Without containerization
|
||||
|
||||
```sh
|
||||
python3 -m venv .venv
|
||||
. .venv/bin/activate
|
||||
pip install -r requirements.lock
|
||||
uvicorn src.main:app --no-access-log --proxy-headers --forwarded-allow-ips '*' --host 0.0.0.0 --port 5758
|
||||
Run the server
|
||||
```
|
||||
python3 main.py
|
||||
```
|
||||
|
||||
### 🛡️ Running behind a reverse proxy
|
||||
Now you can point your reverse proxy to `http://localhost:2944`
|
||||
|
||||
To run the app behind a reverse proxy, ensure that the appropriate proxy headers are added.
|
||||
Below is a sample configuration for NGINX:
|
||||
You can change the bind address and port with the environment variables `RD_BIND` and `RD_PORT`. The default values are `0.0.0.0` and `2944`, respectively.
|
||||
|
||||
```text
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:5758;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
# Redirection
|
||||
Simply replace a urban dictionary url with a Rural Dictionary url from the instance list above.
|
||||
```
|
||||
https://urbandictionary.com/define.php?term=eevee
|
||||
```
|
||||
|
||||
## 🔧 Development
|
||||
becomes
|
||||
|
||||
Install Rye by following
|
||||
the [installation guide](https://rye.astral.sh/guide/installation/).
|
||||
|
||||
Use `rye sync` to install dependencies and required Python version.
|
||||
|
||||
Use `rye run dev` to start development server which will reload on every change to source code.
|
||||
|
||||
Use `rye check --fix` and `rye fmt` to lint and format code. Assumed to be run before each commit
|
||||
to guarantee code quality.
|
||||
|
||||
Use `rye run basedpyright` to ensure typing is correct.
|
||||
|
||||
## 🤝 Support
|
||||
|
||||
Join our [Matrix room](https://mto.vern.cc/#/#cobra-frontends:vern.cc) for support and other
|
||||
things related to Rural Dictionary.
|
||||
|
||||
## 🔗 Redirection
|
||||
|
||||
To use Rural Dictionary, simply replace an Urban Dictionary URL with a Rural Dictionary URL from
|
||||
the instance list above. Auto-redirect browser extension
|
||||
like [Redirector](https://github.com/einaregilsson/Redirector) can be used to achieve this.
|
||||
|
||||
For example, change:
|
||||
|
||||
`https://urbandictionary.com/define.php?term=kin`
|
||||
|
||||
to:
|
||||
|
||||
`https://rd.vern.cc/define.php?term=kin`
|
||||
|
||||
**Note:** More endpoints are supported.
|
||||
|
||||
## 👥 Contributors
|
||||
|
||||
- [thirtysix](https://thirtysix.pw), rewrote project in a more modern libraries stack and
|
||||
implemented missing Urban Dictionary features
|
||||
- [ncts](https://codeberg.org/ncts), added like/dislike counter
|
||||
- [zortazert](https://codeberg.org/zortazert), created the initial Urban Dictionary frontend using
|
||||
JavaScript and helped develop Rural Dictionary
|
||||
|
||||
## 📜 License
|
||||
|
||||
This project is licensed under the AGPLv3+ license - see the [license file](LICENSE) for details.
|
||||
```
|
||||
https://rd.vern.cc/define.php?term=eevee
|
||||
```
|
||||
NOTE: More endpoints are supported
|
||||
|
|
|
|||
|
|
@ -1,7 +1,12 @@
|
|||
version: "3"
|
||||
services:
|
||||
rural-dict:
|
||||
build: .
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "127.0.0.1:5758:5758"
|
||||
- 2944:2944
|
||||
environment:
|
||||
- RD_PORT=2944
|
||||
- RD_BIND=0.0.0.0
|
||||
|
|
|
|||
|
|
@ -1,42 +1,18 @@
|
|||
[
|
||||
{
|
||||
"clearnet": "https://rd.vern.cc",
|
||||
"tor": "http://rd.vernccvbvyi5qhfzyqengccj7lkove6bjot2xhh5kajhwvidqafczrad.onion",
|
||||
"i2p": "http://vern5cxiaufqvhv4hu5ypkvw3tiwvuinae4evdbqzrioql6s2sha.b32.i2p",
|
||||
"country": "US",
|
||||
"owner_name": "~vern",
|
||||
"owner_website": "https://vern.cc"
|
||||
"clearnet": "https://rd.vern.cc",
|
||||
"tor": "http://rd.vernccvbvyi5qhfzyqengccj7lkove6bjot2xhh5kajhwvidqafczrad.onion",
|
||||
"i2p": "http://vern5cxiaufqvhv4hu5ypkvw3tiwvuinae4evdbqzrioql6s2sha.b32.i2p",
|
||||
"country": "US",
|
||||
"owner_name": "~vern",
|
||||
"owner_website": "https://vern.cc"
|
||||
},
|
||||
{
|
||||
"clearnet": "https://rd.bloat.cat",
|
||||
"tor": null,
|
||||
"i2p": null,
|
||||
"country": "DE",
|
||||
"owner_name": "bloatcat",
|
||||
"owner_website": "https://bloat.cat"
|
||||
},
|
||||
{
|
||||
"clearnet": "https://rd.thirtysix.pw",
|
||||
"tor": null,
|
||||
"i2p": null,
|
||||
"country": "NL",
|
||||
"owner_name": "thirtysix",
|
||||
"owner_website": "https://thirtysix.pw"
|
||||
},
|
||||
{
|
||||
"clearnet": "https://rd.cmc.pub",
|
||||
"tor": null,
|
||||
"i2p": null,
|
||||
"country": "US",
|
||||
"owner_name": "~cmc",
|
||||
"owner_website": "https://cmc.pub"
|
||||
},
|
||||
{
|
||||
"clearnet": "https://ruraldictionary.franklyflawless.org",
|
||||
"tor": null,
|
||||
"i2p": null,
|
||||
"country": "DE",
|
||||
"owner_name": "FranklyFlawless",
|
||||
"owner_website": "https://franklyflawless.org"
|
||||
"clearnet": "https://rd.bloat.cat",
|
||||
"tor": null,
|
||||
"i2p": null,
|
||||
"country": "RO",
|
||||
"owner_name": "bloatcat",
|
||||
"owner_website": "https://bloat.cat"
|
||||
}
|
||||
]
|
||||
|
|
|
|||
87
main.py
Normal file
87
main.py
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
#!/usr/bin/env python
|
||||
## Copyright (C) 2023-2025 Skylar Astaroth <cobra@vern.cc>
|
||||
## Copyright (C) 2024 Zubarev Grigoriy <thirtysix@thirtysix.pw>
|
||||
## Copyright (C) 2024 Blair Noctis <ncts@debian.org>
|
||||
##
|
||||
## This file is part of Rural Dictionary (rural-dict)
|
||||
##
|
||||
## rural-dict is free software: you can redistribute it and/or modify it under
|
||||
## the terms of the GNU Affero General Public License as published by the Free
|
||||
## Software Foundation, either version 3 of the License, or (at your option) any
|
||||
## later version.
|
||||
##
|
||||
## This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
## ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
|
||||
## FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License
|
||||
## for more details.
|
||||
##
|
||||
## You should have received a copy of the GNU Affero General Public License
|
||||
## along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
from flask import Flask, render_template, request, redirect
|
||||
import requests
|
||||
import html
|
||||
import re
|
||||
from bs4 import BeautifulSoup
|
||||
from urllib.parse import quote, unquote
|
||||
import os
|
||||
|
||||
def scrape(url):
|
||||
data = requests.get(url)
|
||||
|
||||
our_path = re.sub(r".*://.*/", "/", request.url)
|
||||
path = re.sub(r".*://.*/", "/", data.url)
|
||||
if our_path != path and \
|
||||
quote(unquote(re.sub("[?&=]", "", our_path))) != re.sub("[?&=]", "", path):
|
||||
# this is bad ^
|
||||
return f"REDIRECT {path}"
|
||||
ret = []
|
||||
soup = BeautifulSoup(data.text, "html.parser")
|
||||
|
||||
defs = [(div, div.get('data-defid')) for div in soup.find_all("div") if div.get('data-word')]
|
||||
|
||||
try:
|
||||
votes_data = requests.get(
|
||||
'https://www.urbandictionary.com/api/vote?defids=' + ','.join(defid for (_, defid) in defs) + '&signature=' + soup.body.get('data-vote-signature')
|
||||
).json()['votes']
|
||||
except:
|
||||
votes_data = {}
|
||||
|
||||
for (definition, defid) in defs:
|
||||
word = definition.select("div div h1 a, div div h2 a")[0].get_text()
|
||||
meaning = definition.find(attrs={"class": ["break-words meaning mb-4"]}).decode_contents()
|
||||
example = definition.find(attrs={"class": ["break-words example italic mb-4"]}).decode_contents()
|
||||
contributor = definition.find(attrs={"class": ["contributor font-bold"]})
|
||||
votes_up = votes_data.get(str(defid), {}).get('up')
|
||||
votes_down = votes_data.get(str(defid), {}).get('down')
|
||||
ret.append([defid, word, meaning, example, contributor, votes_up, votes_down])
|
||||
|
||||
pages = soup.find(attrs={"class": ["pagination text-xl text-center"]})
|
||||
if pages == None:
|
||||
pages = ""
|
||||
|
||||
if ret == []:
|
||||
ret = ["SIMILAR"]
|
||||
words = soup.find("ul", attrs={"class": ["mt-5 list-none"]})
|
||||
if words:
|
||||
for word in words.find_all("a"):
|
||||
ret.append(word.get_text())
|
||||
return ret
|
||||
|
||||
return (ret, pages)
|
||||
|
||||
app = Flask(__name__, template_folder="templates", static_folder="static")
|
||||
|
||||
@app.route('/', defaults={'path': ''})
|
||||
@app.route('/<path:path>')
|
||||
def catch_all(path):
|
||||
scraped = scrape(f"https://urbandictionary.com/{re.sub(r'.*://.*/', '/', request.url)}")
|
||||
if type(scraped) == str and scraped.startswith("REDIRECT"):
|
||||
return redirect(scraped.replace("REDIRECT ", ""), 302)
|
||||
elif scraped[0] == "SIMILAR":
|
||||
return render_template('similar.html', similar_words=scraped[1:], term=request.args.get("term"))
|
||||
return render_template('index.html', results=scraped[0], pagination=scraped[1], term=request.args.get("term"))
|
||||
|
||||
if __name__ == '__main__':
|
||||
from waitress import serve
|
||||
serve(app, host=os.environ.get('RD_BIND', "0.0.0.0"), port=int(os.environ.get('RD_PORT', 2944)))
|
||||
|
|
@ -1,84 +0,0 @@
|
|||
[project]
|
||||
name = "rural-dict"
|
||||
version = "1.0.0"
|
||||
description = "Privacy-respecting, NoJS-supporting Urban Dictionary frontend."
|
||||
license = "AGPL-3.0-or-later"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
authors = [
|
||||
{ name = "Zubarev Grigoriy", email = "thirtysix@thirtysix.pw" },
|
||||
{ name = "vlnst", email = "vlnst@bloat.cat" },
|
||||
{ name = "Skylar Astaroth", email = "cobra@vern.cc" },
|
||||
{ name = "zortazert", email = "zortazert@matthewevan.xyz" },
|
||||
]
|
||||
dependencies = [
|
||||
"aiohttp~=3.10.3",
|
||||
"selectolax~=0.3.21",
|
||||
"fastapi~=0.112.1",
|
||||
"uvicorn[standard]~=0.30.6",
|
||||
"jinja2~=3.1.4",
|
||||
]
|
||||
|
||||
[tool.rye]
|
||||
virtual = true
|
||||
managed = true
|
||||
universal = true
|
||||
dev-dependencies = [
|
||||
"basedpyright>=1.16.0",
|
||||
]
|
||||
|
||||
[tool.rye.scripts]
|
||||
dev = """uvicorn src.main:app --reload --reload-include 'src/**/*.py'
|
||||
--reload-include 'templates/**/*.html' --reload-include 'static/**/*.css' --port 5758"""
|
||||
start = """uvicorn src.main:app --no-access-log --proxy-headers
|
||||
--forwarded-allow-ips '*' --host 0.0.0.0 --port 5758"""
|
||||
|
||||
[tool.ruff]
|
||||
target-version = "py312"
|
||||
line-length = 99
|
||||
exclude = [
|
||||
".git",
|
||||
".venv",
|
||||
".idea",
|
||||
".tests",
|
||||
"build",
|
||||
"dist",
|
||||
]
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = [
|
||||
"E", # pycodestyle errors
|
||||
"W", # pycodestyle warnings
|
||||
"F", # pyflakes
|
||||
"I", # isort
|
||||
"N", # pep8-naming
|
||||
"S", # flake8-bandit
|
||||
"B", # flake8-bugbear
|
||||
"G", # flake8-logging-format
|
||||
"C4", # flake8-comprehensions
|
||||
"UP", # pyupgrade
|
||||
"PLC", # pylint conventions
|
||||
"PLE", # pylint errors
|
||||
"SIM", # flake8-simplify
|
||||
"RET", # flake8-return
|
||||
"YTT", # flake8-2020
|
||||
"RUF", # ruff-specific rules
|
||||
"TCH", # flake8-type-checking
|
||||
"PTH", # flake8-use-pathlib
|
||||
"ASYNC", # flake8-async
|
||||
]
|
||||
|
||||
[tool.basedpyright]
|
||||
exclude = [
|
||||
".git",
|
||||
".venv",
|
||||
".idea",
|
||||
".tests",
|
||||
"build",
|
||||
"dist",
|
||||
]
|
||||
typeCheckingMode = "standard"
|
||||
pythonPlatform = "All"
|
||||
pythonVersion = "3.12"
|
||||
reportMissingImports = true
|
||||
reportMissingTypeStubs = false
|
||||
|
|
@ -1,74 +0,0 @@
|
|||
# generated by rye
|
||||
# use `rye lock` or `rye sync` to update this lockfile
|
||||
#
|
||||
# last locked with the following flags:
|
||||
# pre: false
|
||||
# features: []
|
||||
# all-features: false
|
||||
# with-sources: false
|
||||
# generate-hashes: false
|
||||
# universal: true
|
||||
|
||||
aiohappyeyeballs==2.3.6
|
||||
# via aiohttp
|
||||
aiohttp==3.10.3
|
||||
aiosignal==1.3.1
|
||||
# via aiohttp
|
||||
annotated-types==0.7.0
|
||||
# via pydantic
|
||||
anyio==4.4.0
|
||||
# via starlette
|
||||
# via watchfiles
|
||||
attrs==24.2.0
|
||||
# via aiohttp
|
||||
basedpyright==1.16.0
|
||||
click==8.1.7
|
||||
# via uvicorn
|
||||
colorama==0.4.6 ; platform_system == 'Windows' or sys_platform == 'win32'
|
||||
# via click
|
||||
# via uvicorn
|
||||
fastapi==0.112.1
|
||||
frozenlist==1.4.1
|
||||
# via aiohttp
|
||||
# via aiosignal
|
||||
h11==0.14.0
|
||||
# via uvicorn
|
||||
httptools==0.6.4
|
||||
# via uvicorn
|
||||
idna==3.7
|
||||
# via anyio
|
||||
# via yarl
|
||||
jinja2==3.1.4
|
||||
markupsafe==2.1.5
|
||||
# via jinja2
|
||||
multidict==6.0.5
|
||||
# via aiohttp
|
||||
# via yarl
|
||||
nodejs-wheel-binaries==20.16.0
|
||||
# via basedpyright
|
||||
pydantic==2.8.2
|
||||
# via fastapi
|
||||
pydantic-core==2.20.1
|
||||
# via pydantic
|
||||
python-dotenv==1.0.1
|
||||
# via uvicorn
|
||||
pyyaml==6.0.2
|
||||
# via uvicorn
|
||||
selectolax==0.3.33
|
||||
sniffio==1.3.1
|
||||
# via anyio
|
||||
starlette==0.38.2
|
||||
# via fastapi
|
||||
typing-extensions==4.12.2
|
||||
# via fastapi
|
||||
# via pydantic
|
||||
# via pydantic-core
|
||||
uvicorn==0.30.6
|
||||
uvloop==0.21.0 ; platform_python_implementation != 'PyPy' and sys_platform != 'cygwin' and sys_platform != 'win32'
|
||||
# via uvicorn
|
||||
watchfiles==0.23.0
|
||||
# via uvicorn
|
||||
websockets==12.0
|
||||
# via uvicorn
|
||||
yarl==1.9.4
|
||||
# via aiohttp
|
||||
|
|
@ -1,71 +0,0 @@
|
|||
# generated by rye
|
||||
# use `rye lock` or `rye sync` to update this lockfile
|
||||
#
|
||||
# last locked with the following flags:
|
||||
# pre: false
|
||||
# features: []
|
||||
# all-features: false
|
||||
# with-sources: false
|
||||
# generate-hashes: false
|
||||
# universal: true
|
||||
|
||||
aiohappyeyeballs==2.3.6
|
||||
# via aiohttp
|
||||
aiohttp==3.10.3
|
||||
aiosignal==1.3.1
|
||||
# via aiohttp
|
||||
annotated-types==0.7.0
|
||||
# via pydantic
|
||||
anyio==4.4.0
|
||||
# via starlette
|
||||
# via watchfiles
|
||||
attrs==24.2.0
|
||||
# via aiohttp
|
||||
click==8.1.7
|
||||
# via uvicorn
|
||||
colorama==0.4.6 ; platform_system == 'Windows' or sys_platform == 'win32'
|
||||
# via click
|
||||
# via uvicorn
|
||||
fastapi==0.112.1
|
||||
frozenlist==1.4.1
|
||||
# via aiohttp
|
||||
# via aiosignal
|
||||
h11==0.14.0
|
||||
# via uvicorn
|
||||
httptools==0.6.4
|
||||
# via uvicorn
|
||||
idna==3.7
|
||||
# via anyio
|
||||
# via yarl
|
||||
jinja2==3.1.4
|
||||
markupsafe==2.1.5
|
||||
# via jinja2
|
||||
multidict==6.0.5
|
||||
# via aiohttp
|
||||
# via yarl
|
||||
pydantic==2.8.2
|
||||
# via fastapi
|
||||
pydantic-core==2.20.1
|
||||
# via pydantic
|
||||
python-dotenv==1.0.1
|
||||
# via uvicorn
|
||||
pyyaml==6.0.2
|
||||
# via uvicorn
|
||||
selectolax==0.3.33
|
||||
sniffio==1.3.1
|
||||
# via anyio
|
||||
starlette==0.38.2
|
||||
# via fastapi
|
||||
typing-extensions==4.12.2
|
||||
# via fastapi
|
||||
# via pydantic
|
||||
# via pydantic-core
|
||||
uvicorn==0.30.6
|
||||
uvloop==0.21.0 ; platform_python_implementation != 'PyPy' and sys_platform != 'cygwin' and sys_platform != 'win32'
|
||||
# via uvicorn
|
||||
watchfiles==0.23.0
|
||||
# via uvicorn
|
||||
websockets==12.0
|
||||
# via uvicorn
|
||||
yarl==1.9.4
|
||||
# via aiohttp
|
||||
4
requirements.txt
Normal file
4
requirements.txt
Normal file
|
|
@ -0,0 +1,4 @@
|
|||
beautifulsoup4
|
||||
requests
|
||||
flask
|
||||
waitress
|
||||
144
src/main.py
144
src/main.py
|
|
@ -1,144 +0,0 @@
|
|||
## Copyright (C) 2023-2025 Skylar Astaroth <cobra@vern.cc>
|
||||
## Copyright (C) 2024 thirtysix <thirtysix@thirtysix.pw>
|
||||
## Copyright (C) 2024 Blair Noctis <ncts@debian.org>
|
||||
##
|
||||
## This file is part of MeMe
|
||||
##
|
||||
## MeMe is free software: you can redistribute it and/or modify it under the
|
||||
## terms of the GNU Affero General Public License as published by the Free
|
||||
## Software Foundation, either version 3 of the License, or (at your option) any
|
||||
## later version.
|
||||
##
|
||||
## This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
## ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
|
||||
## FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License
|
||||
## for more details.
|
||||
##
|
||||
## You should have received a copy of the GNU Affero General Public License
|
||||
## along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
import re
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import datetime
|
||||
from json import JSONDecodeError
|
||||
|
||||
import aiohttp
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.responses import HTMLResponse, RedirectResponse
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from fastapi.templating import Jinja2Templates
|
||||
from selectolax.parser import HTMLParser, Node
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
"""Establishing an aiohttp ClientSession for the duration of the app's lifecycle."""
|
||||
global session
|
||||
session = aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(10))
|
||||
yield
|
||||
await session.close()
|
||||
|
||||
|
||||
app = FastAPI(lifespan=lifespan, docs_url=None, redoc_url=None)
|
||||
app.mount("/static", StaticFiles(directory="static"), name="static")
|
||||
templates = Jinja2Templates(directory="templates")
|
||||
session: aiohttp.ClientSession = None # pyright: ignore[reportAssignmentType]
|
||||
|
||||
|
||||
def remove_classes(node: Node) -> Node:
|
||||
"""Recursively remove all classes from all nodes."""
|
||||
if "class" in node.attributes:
|
||||
del node.attrs["class"] # pyright: ignore [reportIndexIssue]
|
||||
for child in node.iter():
|
||||
remove_classes(child)
|
||||
return node
|
||||
|
||||
|
||||
@app.get("/{path:path}", response_class=HTMLResponse)
|
||||
async def catch_all(response: Request):
|
||||
"""Handle all routes on Urban Dictionary and perform redirection if necessary."""
|
||||
path_without_host = (
|
||||
f"{response.url.path}{f'?{response.url.query}' if response.url.query else ''}"
|
||||
)
|
||||
url = f"https://www.urbandictionary.com{path_without_host}"
|
||||
term = response.query_params.get("term")
|
||||
|
||||
async with session.get(url) as dict_response:
|
||||
if dict_response.history:
|
||||
return RedirectResponse(str(dict_response.url.relative()), status_code=301)
|
||||
html = await dict_response.text()
|
||||
parser = HTMLParser(html)
|
||||
if dict_response.status != 200:
|
||||
similar_words = None
|
||||
if (try_this := parser.css_first("div.try-these")) is not None:
|
||||
similar_words = [remove_classes(word).html for word in try_this.css("li a")]
|
||||
return templates.TemplateResponse(
|
||||
"404.html",
|
||||
{
|
||||
"request": response,
|
||||
"similar_words": similar_words,
|
||||
"term": term,
|
||||
"site_title": f"Rural Dictionary: {term}",
|
||||
"site_description": (
|
||||
"View on Rural Dictionary, an alternative private "
|
||||
"frontend to Urban Dictionary."
|
||||
),
|
||||
},
|
||||
status_code=404,
|
||||
)
|
||||
|
||||
results = []
|
||||
definitions = parser.css("div[data-defid]")
|
||||
try:
|
||||
thumbs_api_url = (
|
||||
f'https://api.urbandictionary.com/v0/uncacheable?ids='
|
||||
f'{",".join(d.attributes["data-defid"] or "-1" for d in definitions)}'
|
||||
)
|
||||
async with session.get(thumbs_api_url) as thumbs_response:
|
||||
thumbs_json = await thumbs_response.json()
|
||||
thumbs_data = {el["defid"]: el for el in thumbs_json["thumbs"]}
|
||||
except (KeyError, JSONDecodeError, TimeoutError):
|
||||
thumbs_data = {}
|
||||
|
||||
site_description = None
|
||||
for definition in definitions:
|
||||
word = definition.css_first("a.word").text()
|
||||
meaning_node = remove_classes(definition.css_first("div.meaning"))
|
||||
if site_description is None:
|
||||
site_description = re.sub(r"\s+", " ", meaning_node.text(strip=True, separator=" "))
|
||||
meaning = meaning_node.html
|
||||
example = remove_classes(definition.css_first("div.example")).html
|
||||
contributor = remove_classes(definition.css_first("div.contributor")).html
|
||||
definition_id = int(definition.attributes["data-defid"] or "-1")
|
||||
definition_thumbs = thumbs_data.get(definition_id, {})
|
||||
thumbs_up = definition_thumbs.get("up")
|
||||
thumbs_down = definition_thumbs.get("down")
|
||||
results.append(
|
||||
[definition_id, word, meaning, example, contributor, thumbs_up, thumbs_down]
|
||||
)
|
||||
if (pagination := parser.css_first("div.pagination")) is not None:
|
||||
pagination = remove_classes(pagination)
|
||||
pagination.attrs["class"] = "pagination" # pyright: ignore [reportIndexIssue]
|
||||
pagination = pagination.html
|
||||
|
||||
term = term or results[0][1]
|
||||
site_title = "Rural Dictionary"
|
||||
match response.url.path:
|
||||
case "/":
|
||||
# add current date for page with words of the day
|
||||
site_title += f', {datetime.now().strftime("%d %B")}'
|
||||
case "/random.php":
|
||||
term = "Random words"
|
||||
site_title += f": {term}"
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"index.html",
|
||||
{
|
||||
"request": response,
|
||||
"results": results,
|
||||
"pagination": pagination,
|
||||
"term": term,
|
||||
"site_title": site_title,
|
||||
"site_description": site_description,
|
||||
},
|
||||
)
|
||||
|
|
@ -1,49 +1,34 @@
|
|||
body {
|
||||
font-family: DejaVu Sans Mono, monospace;
|
||||
margin: 20px auto;
|
||||
max-width: 800px;
|
||||
line-height: 1.5em;
|
||||
font-size: 1.1em;
|
||||
background-color: #282c34;
|
||||
color: #bbc2cf;
|
||||
padding: 0 10px;
|
||||
hyphens: auto;
|
||||
font-family: DejaVu Sans Mono, monospace;
|
||||
margin:20px auto;
|
||||
max-width:800px;
|
||||
line-height:1.5em;
|
||||
font-size:1.1em;
|
||||
background-color:#282c34;
|
||||
color:#bbc2cf;
|
||||
padding:0 10px;
|
||||
hyphens:auto;
|
||||
}
|
||||
|
||||
img {
|
||||
max-width: 80vw;
|
||||
}
|
||||
|
||||
a {
|
||||
color: #ff6c6b;
|
||||
text-decoration: none;
|
||||
}
|
||||
a:hover {
|
||||
color: #ff6c6b;
|
||||
text-decoration: underline;
|
||||
}
|
||||
.underline-links a {
|
||||
text-decoration: underline;
|
||||
max-width:80vw;
|
||||
}
|
||||
|
||||
a { color:#ff6c6b; text-decoration:none; }
|
||||
a:hover { color:#ff6c6b; text-decoration:underline; }
|
||||
h2 {
|
||||
display: inline;
|
||||
line-height: 1.2;
|
||||
color: #51afef;
|
||||
font-size: 1.2em;
|
||||
display:inline;
|
||||
line-height:1.2;
|
||||
color:#51afef;
|
||||
font-size:1.2em;
|
||||
}
|
||||
|
||||
input {
|
||||
background-color: #282c34;
|
||||
color: #bbc2cf;
|
||||
}
|
||||
|
||||
input { background-color: #282c34; color: #bbc2cf; }
|
||||
.pagination {
|
||||
margin-right: 1ch;
|
||||
text-align: center;
|
||||
margin-right: 1ch;
|
||||
text-align: center;
|
||||
}
|
||||
.pagination ul > li {
|
||||
list-style: none;
|
||||
display: inline-block;
|
||||
padding-left: 1ch;
|
||||
list-style: none;
|
||||
display: inline-block;
|
||||
padding-left: 1ch;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,12 +0,0 @@
|
|||
{% extends "base.html" %} {% block content %}
|
||||
<div style="text-align: center">
|
||||
<h2>Definition not found: {{ term }}</h2>
|
||||
{% if similar_words %}
|
||||
{% for word in similar_words %}
|
||||
<h3 class="underline-links">{{ word | safe }}</h3>
|
||||
{% endfor %}
|
||||
{% else %}
|
||||
<p>There are no similar words. Try correcting your search.</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endblock %}
|
||||
|
|
@ -4,8 +4,8 @@
|
|||
<meta charset="UTF-8" />
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<link rel="stylesheet" href="{{ url_for('static', path='css/main.css') }}" />
|
||||
<link rel="icon" type="image/png" href="{{ url_for('static', path='img/favicon.png') }}" />
|
||||
<link rel="stylesheet" href="{{ url_for('static', filename='css/main.css') }}" />
|
||||
<link rel="icon" type="image/png" href="{{ url_for('static', filename='img/favicon.png') }}" />
|
||||
<title>{{ site_title }}</title>
|
||||
<meta name="description" content="{{ site_description }}" />
|
||||
<!-- The Open Graph Protocol meta tags -->
|
||||
|
|
@ -21,7 +21,7 @@
|
|||
<body>
|
||||
<div style="text-align: center">
|
||||
<a href="/">
|
||||
<img src="{{ url_for('static', path='img/logo.png') }}" alt="logo" />
|
||||
<img src="{{ url_for('static', filename='img/logo.png') }}" alt="logo" />
|
||||
</a>
|
||||
<form id="search" role="search" method="get" action="/define.php">
|
||||
<input
|
||||
|
|
@ -38,7 +38,7 @@
|
|||
</form>
|
||||
<a href="/random.php">Random</a>
|
||||
<br />
|
||||
<a href="https://git.vern.cc/cobra/rural-dict">Source Code</a>
|
||||
<a href="http://git.vern.cc/cobra/rural-dict">Source Code</a>
|
||||
</div>
|
||||
<br />
|
||||
{% block content %}{% endblock %}
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@
|
|||
<a href="/define.php?term={{ word }}">
|
||||
<h2>{{ word }}</h2>
|
||||
</a>
|
||||
<div class="underline-links">
|
||||
<div>
|
||||
<p>{{ meaning | safe }}</p>
|
||||
<p><i>{{ example | safe }}</i></p>
|
||||
</div>
|
||||
|
|
|
|||
15
templates/similar.html
Normal file
15
templates/similar.html
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
{% extends "base.html" %} {% block content %}
|
||||
<div style="text-align: center">
|
||||
<h2>Definition not found: {{ term }}</h2>
|
||||
</div>
|
||||
{% if similar_words %}
|
||||
<p>Similar words:</p>
|
||||
<ul>
|
||||
{% for word in similar_words %}
|
||||
<li><a href="/define.php?term={{ word }}">{{ word }}</a></li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
{% else %}
|
||||
<p>There are no similar words. Try correcting your search.</p>
|
||||
{% endif %}
|
||||
{% endblock %}
|
||||
Loading…
Add table
Add a link
Reference in a new issue