62 lines
1.5 KiB
Markdown
62 lines
1.5 KiB
Markdown
---
|
||
title: "异步任务处理"
|
||
type: concept
|
||
tags: [async, task-queue, django-q]
|
||
sources: [tiktok-pm-python-django-project.md]
|
||
last_updated: 2026-04-14
|
||
---
|
||
|
||
## Definition
|
||
异步任务处理将耗时的操作（如API调用、数据导入）放到后台任务队列中执行，避免HTTP请求超时。
|
||
|
||
## Implementation: Django-Q
|
||
|
||
### Configuration
|
||
```python
|
||
Q_CLUSTER = {
|
||
'name': 'DjangORM',
|
||
'workers': 4,
|
||
'timeout': 360,
|
||
'retry': 120,
|
||
'queue_limit': 50,
|
||
'orm': 'default',
|
||
}
|
||
```
|
||
|
||
### Task Function
|
||
```python
|
||
from django_q.tasks import async_task
|
||
|
||
def trigger_bright_data_task(urls_list):
|
||
# 触发Bright Data API
|
||
job_id = trigger_bright_data_api(urls_list)
|
||
# 异步轮询结果
|
||
async_task('products.tasks.poll_bright_data_result', job_id)
|
||
return job_id
|
||
|
||
def poll_bright_data_result(job_id):
|
||
# 轮询直到任务完成
|
||
while True:
|
||
status = check_job_status(job_id)
|
||
if status == 'completed':
|
||
download_and_import_data()
|
||
break
|
||
time.sleep(30)
|
||
```
|
||
|
||
### Worker
|
||
启动后台worker处理异步任务：
|
||
```bash
|
||
docker compose exec web python manage.py qcluster
|
||
```
|
||
|
||
## Bright Data Integration
|
||
- 触发异步任务获取job_id
|
||
- 每30秒轮询任务状态
|
||
- 任务完成后下载JSON数据并导入数据库
|
||
|
||
## Connections
|
||
- [[异步任务处理]] ← uses ← [[Django]]
|
||
- [[异步任务处理]] ← calls ← [[Bright Data]]
|
||
- [[异步任务处理]] ← runs_in ← [[Docker容器化部署]]
|