Skip to content

Commit a494f50

Browse files
authored
Improve python sdk (#197)
Signed-off-by: Zike Yang <[email protected]>
1 parent 56f867b commit a494f50

23 files changed

+725
-313
lines changed

sdks/fs-python/Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,4 @@ build-image:
44
docker build -t functionstream/fs-python-base .
55

66
test:
7-
pytest -v
7+
PYTHONPATH=. python -m pytest

sdks/fs-python/README.md

Lines changed: 22 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -16,31 +16,31 @@
1616

1717
# FunctionStream Python SDK
1818

19-
FunctionStream SDK is a powerful Python library for building and deploying serverless functions that process messages from Apache Pulsar. It provides a simple yet flexible framework for creating event-driven applications with robust error handling, metrics collection, and resource management.
19+
FunctionStream SDK is a powerful Python library for building and deploying serverless functions that process messages
20+
from Apache Pulsar. It provides a simple yet flexible framework for creating event-driven applications with robust error
21+
handling, metrics collection, and resource management.
2022

2123
## Features
2224

2325
- **Easy Function Development**: Simple API for creating serverless functions
2426
- **Message Processing**: Built-in support for Apache Pulsar message processing
2527
- **Metrics Collection**: Automatic collection of performance metrics
2628
- **Resource Management**: Efficient handling of connections and resources
27-
- **Graceful Shutdown**: Proper cleanup of resources during shutdown
2829
- **Configuration Management**: Flexible configuration through YAML files
29-
- **Schema Validation**: Input and output schema validation
3030
- **Error Handling**: Comprehensive error handling and logging
3131

3232
## Installation
3333

3434
```bash
35-
pip install fs-sdk
35+
pip install function-stream
3636
```
3737

3838
## Quick Start
3939

4040
1. Create a function that processes messages:
4141

4242
```python
43-
from fs_sdk import FSFunction
43+
from function_stream import FSFunction
4444

4545
async def my_process_function(request_data: dict) -> dict:
4646
# Process the request data
@@ -105,6 +105,7 @@ modules:
105105
### FSFunction
106106

107107
The main class for creating serverless functions. It handles:
108+
108109
- Message consumption and processing
109110
- Response generation
110111
- Resource management
@@ -114,6 +115,7 @@ The main class for creating serverless functions. It handles:
114115
### Configuration
115116

116117
The SDK uses YAML configuration files to define:
118+
117119
- Pulsar connection settings
118120
- Module selection
119121
- Topic subscriptions
@@ -123,6 +125,7 @@ The SDK uses YAML configuration files to define:
123125
### Metrics
124126

125127
Built-in metrics collection for:
128+
126129
- Request processing time
127130
- Success/failure rates
128131
- Message throughput
@@ -140,24 +143,24 @@ Check out the `examples` directory for complete examples:
140143
## Best Practices
141144

142145
1. **Error Handling**
143-
- Always handle exceptions in your process functions
144-
- Use proper logging for debugging
145-
- Implement graceful shutdown
146+
- Always handle exceptions in your process functions
147+
- Use proper logging for debugging
148+
- Implement graceful shutdown
146149

147150
2. **Resource Management**
148-
- Close resources properly
149-
- Use context managers when possible
150-
- Monitor resource usage
151+
- Close resources properly
152+
- Use context managers when possible
153+
- Monitor resource usage
151154

152155
3. **Configuration**
153-
- Use environment variables for sensitive data
154-
- Validate configuration values
155-
- Document configuration options
156+
- Use environment variables for sensitive data
157+
- Validate configuration values
158+
- Document configuration options
156159

157160
4. **Testing**
158-
- Write unit tests for your functions
159-
- Test error scenarios
160-
- Validate input/output schemas
161+
- Write unit tests for your functions
162+
- Test error scenarios
163+
- Validate input/output schemas
161164

162165
## Development
163166

@@ -180,13 +183,13 @@ source venv/bin/activate # Linux/Mac
180183
pip install -r requirements.txt
181184
182185
# Install the package in development mode
183-
pip install -e .
186+
python -m pip install -e .
184187
```
185188

186189
### Running Tests
187190

188191
```bash
189-
pytest
192+
make test
190193
```
191194

192195
## Support

sdks/fs-python/examples/Dockerfile

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,8 @@ WORKDIR /function
55
COPY requirements.txt .
66
RUN pip install -r requirements.txt
77

8-
COPY string_function.py .
9-
COPY config.yaml .
10-
COPY package.yaml .
8+
COPY main.py .
119

12-
CMD ["python", "string_function.py"]
10+
RUN chmod +x main.py
11+
12+
CMD ["python", "main.py"]

sdks/fs-python/examples/config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616
# This configuration file defines the settings for the string processing function example.
1717

1818
pulsar:
19-
serviceUrl: "pulsar://192.168.31.80:6650" # Required: URL of the Pulsar broker
19+
serviceUrl: "pulsar://127.0.0.1:6650" # Required: URL of the Pulsar broker
2020
authPlugin: "" # Optional: Authentication plugin class name
2121
authParams: "" # Optional: Authentication parameters
2222

sdks/fs-python/examples/string_function.py renamed to sdks/fs-python/examples/main.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,7 @@
2020
import asyncio
2121
from typing import Dict, Any
2222

23-
from fs_sdk import FSFunction
24-
from fs_sdk.context import FSContext
23+
from function_stream import FSFunction, FSContext
2524

2625
async def string_process_function(context: FSContext, data: Dict[str, Any]) -> Dict[str, Any]:
2726
"""

sdks/fs-python/examples/test_string_function.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,6 @@
2020
import pulsar
2121
import json
2222
import uuid
23-
import time
2423

2524
async def test_string_function():
2625
"""

sdks/fs-python/fs_sdk/__init__.py

Lines changed: 0 additions & 4 deletions
This file was deleted.

sdks/fs-python/fs_sdk/config.py

Lines changed: 0 additions & 63 deletions
This file was deleted.
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
from .config import Config, PulsarConfig, PulsarSourceConfig, SourceSpec, SinkSpec, Metric
2+
from .context import FSContext
3+
from .function import FSFunction
4+
from .metrics import Metrics, MetricsServer
5+
from .module import FSModule
6+
7+
__version__ = "0.6.0rc1"
8+
__all__ = [
9+
# Core classes
10+
"FSFunction",
11+
"FSModule",
12+
13+
# Configuration classes
14+
"Config",
15+
"PulsarConfig",
16+
"PulsarSourceConfig",
17+
"SourceSpec",
18+
"SinkSpec",
19+
"Metric",
20+
21+
# Context and utilities
22+
"FSContext",
23+
24+
# Metrics and monitoring
25+
"Metrics",
26+
"MetricsServer"
27+
]

0 commit comments

Comments
 (0)