OpenExplorer/pointpillars

1---2license: other3tags:4  - heal5  - horizon6  - lidar7---8 9# PointPillars (Car)10 11PointPillars voxelizes point clouds into pillars; `PillarFeatureNet` learns pillar features, which are scattered into pseudo-images via `PointPillarScatter`, then multi-scale feature extraction and 3D box regression are performed by a SECOND-style FPN (`SECONDNeck`) and `PointPillarsHead`; training uses joint supervision with FocalLoss + SmoothL1Loss + direction classification.12 13---14 15## Deployment Metrics16 17### Model Parameters18 19| Model | Model Input | Backbone | Neck | Model Output |20|---|---|---|---|---|21| PointPillars | Single-frame LiDAR point cloud `(N,4)` | PointPillarScatter | SECONDNeck | Car 3D detection boxes `(B,N,cls+reg)` |22 23### Accuracy Metrics24 25| March | Metric | float | calibration | qat | hbm |26| --- | --- | --- | --- | --- | --- |27| J6M | 3D AP (Car) | 0.7731 | 0.7569 | 0.7709 | 0.7707 |28 29> Data measured with `march = March.NASH_M` (J6M) configuration.30>31> HEAL version: heal 0.0.2 / hbdk4-compiler 4.11.11 / horizon_plugin_pytorch 3.3.10.32 33### Performance Metrics34 35> **Performance test methodology**: FPS is measured with single-core eight-thread; Latency is measured with single-core single-thread; Memory is peak DDR usage.36 37| March | latency (ms) | fps | Memory Usage (MB) |38|---|---|---|---|39| J6M | 22.52 | 213.58 | 55.90 |40| J6P | 20.43 | 344.88 | 55.90 |41| J6B | 1650.03 | 1.99 | 53.00 |42 43---44 45## Model Overview46 47### Core Design48 49PointPillars voxelizes point clouds into pillars; `PillarFeatureNet` learns pillar features, which are scattered into pseudo-images via `PointPillarScatter`, then multi-scale feature extraction and 3D box regression are performed by a SECOND-style FPN (`SECONDNeck`) and `PointPillarsHead`; training uses joint supervision with FocalLoss + SmoothL1Loss + direction classification.50 51- **Task type**: 3D object detection (LiDAR point cloud 3D Object Detection).52- **backbone**: `PointPillarScatter` (scatters 64-dim pillar features learned by `PillarFeatureNet` into `H×W×64` pseudo-image feature maps by coordinates, `use_horizon_pillar_scatter=True`).53- **neck**: `SECONDNeck` (SECOND-style FPN: downsample layer channels `[64, 128, 256]`, strides `[2, 2, 2]`; upsample layer channels `[128, 128, 128]`, strides `[1, 2, 4]`; outputs three-scale features concatenated to 384 channels).54- **Point cloud preprocessing**: `PointPillarsPreProcess` (voxelization: `pc_range=[0, -39.68, -3, 69.12, 39.68, 1]`, `voxel_size=[0.16, 0.16, 4]`, max 100 points per voxel, max 12000 voxels).55- **Feature extraction**: `PillarFeatureNet` (`num_filters=(64)`, 4-dim input, MLP + max-pool to 64-dim pillar features).56- **Detection head**: `PointPillarsHead` (`in_channels=384`, `use_direction_classifier=True`, outputs classification + box regression + direction classification).57- **Anchor generation**: `Anchor3DGeneratorStride`: Car anchor size `1.6×3.9×1.56`, stride `[0.32, 0.32, 0.0]`, offset `[0.16, -39.52, -1.78]`, rotation angles `[0, 1.57]`; match threshold `0.6`, mismatch threshold `0.45`.58- **Post-processing**: `PointPillarsPostProcess`: NMS (`nms_iou_threshold=0.5`, `score_threshold=0.4`, `nms_pre_max_size=1000`, `nms_post_max_size=300`, `max_per_img=100`).59- **Loss**: `FocalLoss` (classification, `alpha=0.25, gamma=2.0, weight=1.0`) + `SmoothL1Loss` (box regression, `beta=1/9, weight=2.0`) + `CrossEntropyLoss` (direction, `weight=0.2`).60- **Model input**: Single-frame LiDAR point cloud, shape `(N, 4)`, N = number of points (deployment input padded to `150000` points), 4 dims = `[x, y, z, intensity]`.61- **Model output**: Car 3D boxes per frame (`x, y, z, w, l, h, θ`) + score + direction.62- **Classes**: `["Car"]`.63 64**Deployment notes**: The PointPillars deployment graph includes full voxelization + feature extraction + backbone + head + post-processing. HBIR export enables `enable_vpu=True`; compilation uses `input_source=["ddr"]` (point cloud read from DDR, not pyramid image input), unlike image-based tasks. This task has no subgraph export; only one `deploy.py`.65 66### Official Repo and Paper67 68Official repo: https://github.com/nutonomy/second.pytorch69Paper: https://arxiv.org/abs/1812.05784v170 71### Reference72 73For more J6 chip deployment details, see https://developer.horizon.auto/blog/1408674