新增wiki命令文件
This commit is contained in:
61
wiki/concepts/异步任务处理.md
Normal file
61
wiki/concepts/异步任务处理.md
Normal file
@@ -0,0 +1,61 @@
|
||||
---
|
||||
title: "异步任务处理"
|
||||
type: concept
|
||||
tags: [async, task-queue, django-q]
|
||||
sources: [tiktok-pm-python-django-project.md]
|
||||
last_updated: 2026-04-14
|
||||
---
|
||||
|
||||
## Definition
|
||||
异步任务处理将耗时的操作(如API调用、数据导入)放到后台任务队列中执行,避免HTTP请求超时。
|
||||
|
||||
## Implementation: Django-Q
|
||||
|
||||
### Configuration
|
||||
```python
|
||||
Q_CLUSTER = {
|
||||
'name': 'DjangORM',
|
||||
'workers': 4,
|
||||
'timeout': 360,
|
||||
'retry': 120,
|
||||
'queue_limit': 50,
|
||||
'orm': 'default',
|
||||
}
|
||||
```
|
||||
|
||||
### Task Function
|
||||
```python
|
||||
from django_q.tasks import async_task
|
||||
|
||||
def trigger_bright_data_task(urls_list):
|
||||
# 触发Bright Data API
|
||||
job_id = trigger_bright_data_api(urls_list)
|
||||
# 异步轮询结果
|
||||
async_task('products.tasks.poll_bright_data_result', job_id)
|
||||
return job_id
|
||||
|
||||
def poll_bright_data_result(job_id):
|
||||
# 轮询直到任务完成
|
||||
while True:
|
||||
status = check_job_status(job_id)
|
||||
if status == 'completed':
|
||||
download_and_import_data()
|
||||
break
|
||||
time.sleep(30)
|
||||
```
|
||||
|
||||
### Worker
|
||||
启动后台worker处理异步任务:
|
||||
```bash
|
||||
docker compose exec web python manage.py qcluster
|
||||
```
|
||||
|
||||
## Bright Data Integration
|
||||
- 触发异步任务获取job_id
|
||||
- 每30秒轮询任务状态
|
||||
- 任务完成后下载JSON数据并导入数据库
|
||||
|
||||
## Connections
|
||||
- [[异步任务处理]] ← uses ← [[Django]]
|
||||
- [[异步任务处理]] ← calls ← [[Bright Data]]
|
||||
- [[异步任务处理]] ← runs_in ← [[Docker容器化部署]]
|
||||
Reference in New Issue
Block a user