Update nexus wiki content
This commit is contained in:
41
wiki/entities/Snowflake.md
Normal file
41
wiki/entities/Snowflake.md
Normal file
@@ -0,0 +1,41 @@
|
||||
---
|
||||
title: "Snowflake"
|
||||
type: entity
|
||||
tags: [data-engineering, cloud-data-warehouse, lakehouse, SaaS]
|
||||
sources: [engineering-data-engineer]
|
||||
last_updated: 2026-05-02
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Snowflake 是云原生数据仓库和数据湖平台,支持多集群共享数据架构,提供独立扩展存储和计算的能力。Data Engineer Agent 将 Snowflake 作为 Medallion Architecture Gold 层的重要消费平台。
|
||||
|
||||
## Key Features
|
||||
|
||||
### Dynamic Tables
|
||||
- 声明式管道:定义目标状态,Snowflake 自动计算增量更新
|
||||
- 无需管理 `MERGE INTO` 或调度逻辑——Snowflake 处理增量计算
|
||||
- 类似于 dbt 的 incremental models,但由平台原生支持
|
||||
|
||||
### Snowpark
|
||||
- 在 Snowflake 内运行 Python、Scala、Java 代码
|
||||
- 支持自定义函数(UDFs)和存储过程
|
||||
- 将数据处理逻辑下沉到 Snowflake 内部,减少数据传输
|
||||
|
||||
### Data Sharing
|
||||
- Secure Data Sharing:跨账户即时共享数据,无需复制
|
||||
- Provider 和 Consumer 账户独立计费
|
||||
|
||||
### Cost Optimization
|
||||
- 每一列自动压缩存储
|
||||
- 按查询计费的 Serverless SQL(无需预留容量)
|
||||
- Virtual Warehouse 独立扩缩容
|
||||
|
||||
## Snowflake Connectors
|
||||
- **Snowflake Connector for Spark**:Spark DataFrame ↔ Snowflake 互操作
|
||||
- **Snowflake Connector for Kafka**:Kafka → Snowflake 实时摄取
|
||||
- **Native Ingest**:Python/Go/Java SDK 直接摄取
|
||||
|
||||
## Related Concepts
|
||||
- [[Medallion Architecture]](Snowflake 可作为 Gold 层)
|
||||
- [[dbt]](dbt Snowflake adapter 广泛使用)
|
||||
Reference in New Issue
Block a user