TransBiolab — A Real-World Multi-View Dataset of Cluttered Transparent Biomedical Objects

§ 3.1

Objects & Capture Setup

15 laboratory objects organized into five functional groups that frequently appear in biomedical manipulation workflows.

An overview of the proposed TransBiolab dataset, featuring multi-object and multi-view characteristics. Frames denoted by red rectangles indicate higher levels of clutter and occlusion.

TransBiolab contains objects from five groups: well plates (6-, 12-, 24-, 96-well), cell culture dishes (35mm, 60mm, 90mm), liquid-handling objects (15ml and 50ml centrifuge tubes, pipette, tube rack), cell culture flasks (25ml, 75ml, 125ml), and a 1L bioreactor.

Seven objects are fully transparent, six combine transparent bodies with opaque caps or bases, and two are opaque auxiliary tools. This composition reflects how transparent vessels coexist with supporting tools in real laboratory workcells.

Data are collected with an Intel RealSense D435i RGB-D camera at 1280×720 resolution and 30 FPS, mounted on a 7-DoF Franka Emika Panda robot arm that follows calibrated multi-view trajectories around the scene.

15 TransBiolab objects — Fig. 1: Rendering views of the 15 objects included in TransBiolab.

Well Plates

5 6-well plate

6 12-well plate

7 24-well plate

18 96-well plate

Cell Culture Dishes

9 35mm dish

3 60mm dish

11 90mm dish

Liquid Handling

13 15ml centrifuge tube

14 50ml centrifuge tube

1 Pipette

2 Tube-rack

Flasks & Bioreactor

15 25ml flask

16 75ml flask

17 125ml flask

19 Grex 1L bioreactor

§ 3.2

Scene Construction & Capture

Calibrated Multi-View Trajectory

Each scene is captured as an ordered multi-view sequence by the robot-mounted camera following a calibrated hemispheric path. This provides per-frame camera intrinsics and extrinsics, supporting both discrete-view evaluation and future multi-view or video-based methods.

For benchmarking, we uniformly sample 10 equally spaced viewpoints from each trajectory, covering elevation angles from -22° to -62° and varying azimuth angles.

Fig. 2: Robot-mounted capture trajectory with camera pose visualization.

11 Arrangement Patterns

Scenes are constructed from 11 arrangement patterns designed around real laboratory procedures: sample preparation, liquid handling, mixing, sampling, transport, and cell expansion. Patterns range from moderate clutter (4–5 objects) to dense scenes (9–12 objects) with heavy mutual occlusion.

Some patterns emphasize repeated instances from the same category (e.g., multiple tubes), while others mix several functional groups and include opaque auxiliary tools.

Background & Lighting Variations

Controlled scenes are varied by four tabletop backgrounds (white tablecloth, grey tablecloth, bare wood, magazine-covered) and two lighting settings (top light ~190 lx, side light ~75 lx). These factors change contrast, reflections, and occlusion cues without altering scene composition.

Background and lighting variations — Fig. 4: Four backgrounds and two lighting settings.

Fig. 6: Scene-object distribution matrix and initial scene-disjoint training/test partition. ← Scroll horizontally to view full detail →

Fig. 8a: Compact metadata visualization — objectwise viewpoint coverage and the released 6D annotation distributions over translation and rotation axes.

Fig. 8b: 6D pose annotation distributions across translation and rotation axes.

📁 Scene Directory Structure

scene_XXXXXX/
├── rgb/                             # 1280×720 color images
│   ├── 000000.png  …  NNNNNN.png
├── depth/                           # 16-bit depth maps (mm)
│   └── …
├── mask/                            # Full projected masks (incl. occluded)
│   ├── {frame}_{obj_idx}.png
├── mask_visib/                      # Visible-only masks
│   └── …
├── scene_camera.json                # Per-frame K matrix + depth_scale
├── scene_gt.json                    # 6D pose (R, t, obj_id) per instance
└── scene_gt_info.json               # Visibility fraction + bbox

§ 4

Benchmark Results

Evaluation protocols for FoundationPose (6D pose estimation) and SAM 3 (open-set segmentation), stratified by object category, scene clutter, and viewpoint.

4.1 FoundationPose for 6D Pose Estimation

For each target instance, we load the RGB-D frame, visible mask, camera intrinsics, and CAD model, then run the standard register-refine pipeline with 15 refinement iterations. We report ADD, ADD-S, and AUC up to 0.1m.

FoundationPose

6D Pose Estimation

80.8% ADD-S AUC

ADD AUC53.9%

Mean ADD-S23.2 mm

Instances Evaluated1,892

SAM 3

Open-Set Segmentation

68.5% mIoU

Match Rate84.8%

Matched mF10.881

Instances Evaluated1,247

Table 2: FoundationPose Results by Object Category

Name	ADD-S AUC	ADD AUC	ADD-S (mm)
Pipette	93.0	88.2	10.8
Tube-rack	94.1	46.2	9.9
60mm dish	90.1	56.9	9.9
6-well plate	91.7	49.2	8.6
12-well plate	91.9	44.2	8.1
24-well plate	93.4	52.8	6.6
35mm dish	83.8	66.7	17.0
90mm dish	90.6	37.6	11.0
15ml centrifuge tube	75.4	64.4	24.8
50ml centrifuge tube	81.0	58.7	19.7
25ml flask	77.7	57.8	38.1
75ml flask	54.2	35.0	53.2
125ml flask	41.0	28.1	75.1
96-well plate	91.6	53.5	8.6
Grex 1L bioreactor	72.9	45.4	31.9
Overall	80.8	53.9	23.2

Table 3: FoundationPose by Geometric Family

Shape	Representative Objects	ADD-S AUC	ADD-S (mm)
Plate-like	6/12/24/96-well plate	92.2	7.9
Rack / elongated	Pipette, tube-rack	93.6	10.4
Dish-like	35/60/90mm dish	90.4	10.4
Tube-like	15/50ml centrifuge tube	78.2	22.2
Bottle-like	25/75ml flask	65.9	45.6
Large / irregular	125ml flask, Grex 1L bioreactor	56.9	53.5

Table 4: FoundationPose Across 10 Sampled Viewpoints

View	Elev.	Azim.	ADD-S AUC	ADD AUC	ADD-S (mm)
VP1	-22.2°	-68.1°	72.2	47.2	38.7
VP2	-24.6°	-99.0°	72.2	43.2	48.4
VP3	-27.3°	-74.6°	74.0	41.6	28.1
VP4	-32.6°	-86.1°	73.9	48.7	30.1
VP5	-40.9°	-85.0°	79.9	54.1	21.3
VP6	-51.1°	-86.2°	88.0	60.3	12.2
VP7	-51.9°	-103.1°	84.7	56.3	16.1
VP8	-53.4°	-75.2°	87.1	60.3	13.3
VP9	-58.9°	-83.4°	87.6	64.8	12.5
VP10	-62.0°	-75.6°	87.8	61.5	12.5

💡

Viewpoint effect: Steep views (>45°) achieve 85–88% ADD-S AUC (mean 12.5mm), while shallow views (<33°) only reach 72–74% (mean 30–48mm). Mid-to-steep viewpoints expose larger top surfaces and clearer contours of transparent containers.

Scene Complexity Analysis

FoundationPose results by scene complexity

Fig. 9: FoundationPose ADD-S AUC grouped by total visible objects (top) and distinct categories (bottom). Performance is dominated by object geometry rather than scene density.

4.2 SAM 3 for Open-Set Segmentation

SAM 3 is queried with text prompts per object category. Predicted masks are greedily matched to GT instances by IoU. Unmatched GT masks receive IoU = 0.

Table 5: SAM 3 Results by Object Category

Name	Match Rate	mIoU	mF1
Pipette	89.9%	0.718	0.884
Tube-rack	90.0%	0.640	0.828
60mm dish	95.5%	0.818	0.922
6-well plate	85.0%	0.766	0.946
12-well plate	87.8%	0.797	0.951
24-well plate	87.8%	0.802	0.953
35mm dish	83.3%	0.720	0.900
90mm dish	97.1%	0.845	0.925
15ml centrifuge tube	71.7%	0.484	0.795
50ml centrifuge tube	75.6%	0.495	0.780
25ml flask	91.3%	0.735	0.883
75ml flask	77.1%	0.574	0.829
125ml flask	78.6%	0.587	0.799
96-well plate	81.4%	0.721	0.932
Grex 1L bioreactor	100.0%	0.914	0.954
Overall	84.8%	0.685	0.881

Table 6: SAM 3 by Geometric Family

Shape	Representative Objects	mIoU	Match Rate
Plate-like	6/12/24/96-well plate & 35/60/90mm dish	0.789	89.2%
Bottle-like	25/75/125ml flask, tube-rack, Grex 1L	0.700	86.3%
Tube-like	15/50ml centrifuge tube & pipette	0.536	76.1%

Table 7: SAM 3 Across 10 Sampled Viewpoints

View	Elev.	Azim.	mIoU	Match Rate	mF1
VP1	-22.2°	-68.1°	0.681	88.0%	0.856
VP2	-24.6°	-99.0°	0.681	91.1%	0.837
VP3	-27.3°	-74.6°	0.684	89.6%	0.847
VP4	-32.6°	-86.1°	0.685	86.3%	0.874
VP5	-40.9°	-85.0°	0.687	85.5%	0.880
VP6	-51.1°	-86.2°	0.693	82.4%	0.905
VP7	-51.9°	-103.1°	0.682	84.0%	0.886
VP8	-53.4°	-75.2°	0.692	82.4%	0.905
VP9	-58.9°	-83.4°	0.609	71.2%	0.919
VP10	-62.0°	-75.6°	0.583	68.0%	0.915

💡

SAM 3 viewpoint pattern: From VP1 to VP8, mIoU remains stable (0.681–0.693). The main degradation appears at steepest views — VP9 and VP10 drop to 0.609 and 0.583 mIoU, with match rates falling to 71.2% and 68.0%. Failure at steep views is driven by missed detections rather than poor mask quality (mF1 stays high at 0.919/0.915).

Rebuttal

Additional Baselines & Analyses

We thank all reviewers for the constructive comments. We address the main concerns below. TBiolab-HO denotes the 10 held-out lab scenes.

P1. Additional Baselines

To Reviewers zrnj, 5Exs, and TgfS, we added TransLab, ClearGrasp, DA3, and MegaPose-6D, while keeping SAM3 and FoundationPose.

Segmentation

Dataset-defined masks are used; this mask-level protocol complements the paper's instance-matching SAM3 protocol. All metrics are higher-is-better.

Dataset	Method	Target	IoU	Precision	Recall	F1
TransBiolab	SAM3	object mask	0.662	0.881	0.703	0.764
TransBiolab	TransLab	object mask	0.363	0.586	0.653	0.496
ClearPose	SAM3	object mask	0.699	0.885	0.744	0.789
ClearPose	TransLab	object mask	0.491	0.569	0.853	0.632
Trans10K	SAM3	object mask	0.704	0.821	0.827	0.824
Trans10K	TransLab	object mask	0.812	0.874	0.916	0.881
TBiolab-HO	SAM3	object mask	0.477	0.656	0.648	0.632
TBiolab-HO	TransLab	object mask	0.344	0.475	0.665	0.491

↘

This verifies domain specificity: TransLab excels on Trans10K but drops on TransBiolab and TBiolab-HO; SAM3 is lower on TransBiolab than ClearPose, so labware clutter remains unsaturated.

Depth

DA3 uses metric depth, no test-time scale or shift alignment. All metrics are lower-is-better.

Dataset	Split	Method	Reference	AbsRel	RMSE (m)	MAE (m)	Raw missing
TransBiolab	all	DA3	depth GT	0.371	0.224	0.220	0.349
TransBiolab	all	ClearGrasp	depth GT	0.393	0.473	0.349	0.313
ClearPose	all	DA3	depth GT	0.161	0.221	0.141	0.390
ClearPose	all	ClearGrasp	depth GT	0.327	0.404	0.276	0.299
TransBiolab	held-out	DA3	depth GT	0.302	0.180	0.174	0.189
TransBiolab	held-out	ClearGrasp	depth GT	0.179	0.132	0.086	0.178

↘

Compared with ClearPose-all, both DA3 and ClearGrasp have higher object-region depth errors on TransBiolab-all, suggesting that TransBiolab remains challenging on depth estimation and completion tasks.

6D Pose

FoundationPose uses RGB-D, GT mask, CAD; MegaPose-6D uses RGB, GT bbox, CAD, no depth. AUC and Success are higher-is-better; Mean error is lower-is-better.

Dataset	Split	Method	Input	ADD-S AUC	ADD AUC	ADD-S Mean	Success
TransBiolab	all	FoundationPose	RGB-D, GT mask, CAD	80.8	53.9	23.2 mm	67.86%
ClearPose	all	FoundationPose	RGB-D, GT mask, CAD	91.05	49.26	9.77 mm	89.53%
TransBiolab	all	MegaPose-6D	RGB, GT bbox, CAD, no depth	31.76	31.49	67.76 mm	34.14%
ClearPose	all	MegaPose-6D	RGB, GT bbox, CAD, no depth	39.84	12.40	64.29 mm	25.00%
TransBiolab	held-out	FoundationPose	RGB-D, GT mask, CAD	75.35	47.06	34.53 mm	62.31%
TransBiolab	held-out	MegaPose-6D	RGB, GT bbox, CAD, no depth	55.50	25.17	96.46 mm	26.59%

↘

This verifies pose remains hard beyond depth artifacts: FoundationPose drops on TransBiolab and TBiolab-HO, while depth-free MegaPose-6D remains low, showing domain and geometry gaps.

P2. Annotation Reliability

To Reviewers W5GB, TgfS, and 5Exs, we added a geometry-consistency audit over CAD, poses, masks, intrinsics, and trajectories; it measures mutual consistency, not independent metrology.

Pose-mask consistency	visible-mask IoU: mean 0.949; median 0.971
Same-view reprojection	median symmetric contour error: 0.62 px
Cross-view consistency	transferred median contour error: 0.91 px

The 95th-percentile contour error is below 1 px for both audits. We will add overlays and the protocol.

P3. Held-out, Stats, and Redundancy

To Reviewers jqnR, 5Exs, TgfS, and W5GB, held-out scenes above have new layouts, more distractors, and different backgrounds and lighting.

Release size	98 scenes, 161,315 frames
Typical length	about 1,646 frames per scene
Scene setup	each scene has more than 3 objects
>3 in-frame objects	161,059 frames, 99.84%
exactly 3 in-frame objects	256 frames, 0.16%

Exactly-3-object frames occur briefly when viewpoint motion moves objects out of frame; we retain them for continuity.

P4. Robot Experiment

To Reviewers W5GB, 5Exs, and TgfS, this is system-level manipulation, not a proxy for 6D pose accuracy. Parallel-jaw grasping is low-dimensional and error-tolerant, while LinkerHand needs affordance and grasp-type selection, high-dimensional joint mapping, and contact-rich closure.

Its lower success reflects grasp synthesis and control complexity, not only pose error.

P5. Minor Revisions

To Reviewers jqnR, 5Exs, and TgfS, we will fix captions, crop Fig. 1, enlarge figure and table fonts, and reformat tables.

We will clarify that 10 viewpoints are diagnostics while full trajectories are released, and 1,300-1,800 frames follows about 50s 30FPS capture with timing variation.

Qualitative

Visualization Comparison

Side-by-side SAM 3 segmentation masks and FoundationPose 3D bounding box projections on complex multi-object scenes.

1 / 20 Scene 000013 · Frame 000102 · VP01

§ 4.3

Real Robot Experiment

The system first obtains an object mask from segmentation, then estimates a target 6D pose with FoundationPose, transforms this pose to the robot base through hand-eye calibration, and finally plans grasp and placement motions through ROS/MoveIt.

🦾 Franka Parallel-Jaw

65.3% 98 / 150 trials

🤖 LinkerHand 10-DoF

56.67% 85 / 150 trials

Fig. 10: Real robot platform used for cluttered transparent-object manipulation experiments.

🎬 Robot Manipulation Demo

Summary

Key Findings

01

Plate & Dish Objects Excel

Flat plate-shaped objects achieve 90–94% ADD-S AUC and 0.789 mIoU. Their stable contours benefit both pose estimation and segmentation.

02

Large Transparent Bottles Struggle

75ml and 125ml flasks score as low as 41–54% ADD-S AUC. Large transparent volumes cause severe depth and feature ambiguity.

03

Viewpoint Matters for Pose

FoundationPose gains +15.6% AUC from shallow (22°) to steep (62°) views. SAM 3 shows the opposite: degradation at steepest views due to missed detections.

04

Tube-like Objects Are Hardest for SAM 3

Tubes and pipettes achieve only 0.536 mIoU with 76.1% match rate. Thin cylindrical shapes with weak boundaries challenge text-prompted segmentation.

05

Symmetry Exposes ADD vs ADD-S Gap

For symmetric objects the 26.9% gap between ADD-S and ADD confirms that transparent objects need symmetry-aware metrics.

06

Clutter Interacts with Geometry

Scene density alone does not predict difficulty. Performance depends on object geometry × symmetry × visibility, exactly what TransBiolab is designed to surface.