# mlsystem

**Repository Path**: luyanfei/mlsystem

## Basic Information

- **Project Name**: mlsystem
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-06-02
- **Last Updated**: 2026-06-03

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Design a Machine Learning System
## 环境配置
Setup a virtual envrionment by running
```
python -m venv myvenv
source myvenv/bin/activate
```

Install all the requirements by running
```
pip install -r requirements.txt
```
## minio
```
docker run --name minio -d -p 9000:9000 -p 9091:9091 -v minio_volume:/data -e "MINIO_ROOT_USER=minio" -e "MINIO_ROOT_PASSWORD=minio123" minio/minio:latest server /data --console-address :9091
```

## Mlflow
Create an empty bucket in MinIO called mlflow-datasets.

Start mlflow server:
```
mlflow server --port 5000 --host 0.0.0.0
```

## Feast
### 启动Redis服务
```
docker run --name redis1 -d -p 6379:6379 redis:8.6.1 redis-server --requirepass "jGFUi48jvD"
```
Redis的密码要与feature_store.yaml中的一致。

### minio buckets
Also move the datasets in feast/feature_data to minio bucket feature-data-sets.
Create minio bucket feature-registry.

### environment variables
Before running the feast apply command please set the following environment variables
```
export AWS_SECRET_ACCESS_KEY=minio123 
export AWS_ACCESS_KEY_ID=minio
export FEAST_S3_ENDPOINT_URL=http://localhost:9000
export AWS_ENDPOINT_URL=http://localhost:9000
export AWS_S3_ADDRESSING_STYLE=path
```

### feast apply
```
feast apply
```

### Feast materialization
You can run Feast materialization by running the following command
```
feast materialize "2022-09-16T00:00:00" "2023-09-17T00:00:00"
```

### retrieve features

Once feast apply and feast materialize have run you can try retrieving historicalfeatures by running retrieve_features.py, and online features by running retrieve_online_features.py.   

### Feature server
Start feature server:
```
feast serve
```
Test with curl:
```
curl -X POST \
  "http://localhost:6566/get-online-features" \
  -H "Content-Type: application/json" \
  -d '{
  "features": [
    "demographic:Native_country",
    "demographic:Sex",
    "demographic:Race"
  ],
  "entities": {
    "user_id": [
        "9f2ac416-06e1-44a0-87bd-d4787c85bf66"
    ]
  }
}'
```

### feast ui
```
feast ui
```

### To move the feature store yaml to Minio
Upload feature_store.yaml to feature-registry bucket.

## bentoml
### Downloading the MLflow model to the local Bento store
```
python download_model.py
```
查看本地的model:
```
bentoml models list
```

### 本地测试运行

#### 1. 构建 Bento
```bash
cd bentoml
bentoml build -f bento/bentofile.yaml
```

#### 2. 启动服务
```bash
ENV_NAME=production bentoml serve income_classifier_service:latest --port 3000
```

#### 3. 测试预测接口

完整的 curl 命令示例：

```bash
curl -X POST http://localhost:3000/predict \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {
      "user_id": "test123",
      "age": 35,
      "workclass": "Private",
      "fnlwgt": 200000,
      "education": "Bachelors",
      "education_num": 13,
      "marital_status": "Married-civ-spouse",
      "occupation": "Exec-managerial",
      "relationship": "Husband",
      "race": "White",
      "sex": "Male",
      "capital_gain": 0,
      "capital_loss": 0,
      "hours_per_week": 40,
      "native_country": "United-States"
    }
  }'
```

预期返回结果：
```json
{"income_category": "<=50K", "user_id": "test123"}
```

更多测试示例：

```bash
# 示例 2：高收入预测
curl -X POST http://localhost:3000/predict \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {
      "user_id": "user456",
      "age": 45,
      "workclass": "Self-emp-not-inc",
      "fnlwgt": 150000,
      "education": "Masters",
      "education_num": 14,
      "marital_status": "Never-married",
      "occupation": "Prof-specialty",
      "relationship": "Not-in-family",
      "race": "Asian-Pac-Islander",
      "sex": "Female",
      "capital_gain": 50000,
      "capital_loss": 0,
      "hours_per_week": 50,
      "native_country": "United-States"
    }
  }'

# 示例 3：使用 wget
wget --post-data='{"inputs":{"user_id":"test789","age":28,"workclass":"Private","fnlwgt":180000,"education":"HS-grad","education_num":9,"marital_status":"Divorced","occupation":"Craft-repair","relationship":"Unmarried","race":"Black","sex":"Male","capital_gain":0,"capital_loss":0,"hours_per_week":35,"native_country":"United-States"}}' \
  --header='Content-Type: application/json' \
  http://localhost:3000/predict \
  -O - -q
```

#### 4. 查看服务文档
访问 Swagger UI 文档：
```
http://localhost:3000
```

#### 5. 容器化部署（可选）
```bash
# 查看可用的 bento 版本
bentoml list

# 容器化 bento（使用最新版本）
DOCKER_BUILDKIT=0 bentoml containerize income_classifier_service:latest

# 或者使用特定版本
DOCKER_BUILDKIT=0 bentoml containerize income_classifier_service:ibzp72s6rgrtacdr

# 运行容器
docker run --rm -p 3000:3000 income_classifier_service:ibzp72s6rgrtacdr

# 测试容器化的服务
curl -X POST http://localhost:3000/predict \
  -H "Content-Type: application/json" \
  -d '{"inputs": {"user_id": "test123", "age": 35, "workclass": "Private", "fnlwgt": 200000, "education": "Bachelors", "education_num": 13, "marital_status": "Married-civ-spouse", "occupation": "Exec-managerial", "relationship": "Husband", "race": "White", "sex": "Male", "capital_gain": 0, "capital_loss": 0, "hours_per_week": 40, "native_country": "United-States"}}'
```

**注意**：如果你的系统没有安装 Docker BuildKit，请使用 `DOCKER_BUILDKIT=0` 环境变量来禁用 BuildKit。

### 注意事项

1. **环境变量**：确保设置了 `ENV_NAME=production` 以使用正确的配置文件
2. **依赖服务**：确保以下服务正在运行：
   - MLflow server (端口 5000)
   - Redis (端口 6379)
   - MinIO (端口 9000)
3. **模型别名**：确保 MLflow 模型已设置 `champion` 别名
4. **API 格式**：请求数据必须包装在 `inputs` 字段中