Feature Missing: CoCo and other export features on project created with sam

[debpalash]

Is your feature request related to a problem? Please describe.
CoCo and other export features are missing when a project created with sam according to label_studio-ml-backend/examples/segment_anything_model

Describe the solution you'd like
To be able to export the instance segment annotations for yolov8

Describe alternatives you've considered
converting json but it doesn't support segmentation on export popup

Additional context
Project Label Setup

<View>
  <Image name="image" value="$image" zoom="true"/>
  <BrushLabels name="tag" toName="image">
  	<Label value="bicycles" background="#FF0000"/>
  	<Label value="boats" background="#FF0000"/>
  	<Label value="bridges" background="#FF0000"/>
  	<Label value="buses" background="#0d14d3"/>
  	<Label value="chimneys" background="#0d14d3"/>
  	<Label value="crosswalks" background="#0d14d3"/>
  	<Label value="fire hydrants" background="#0cc0ed"/>
  	<Label value="motorcycles" background="#0d14d3"/>
  	<Label value="parking meters" background="#0d14d3"/>
  	<Label value="stairs" background="#0d14d3"/>
  	<Label value="taxis" background="#0d14d3"/>
  	<Label value="tractors" background="#0d14d3"/>
  	<Label value="traffic lights" background="#0d14d3"/>
  	<Label value="vehicles" background="#0d14d3"/>
  </BrushLabels>
  <KeyPointLabels name="tag2" toName="image">
    <Label value="bicycles" smart="true" background="#bb5fec" showInline="true"/>
    <Label value="boats" smart="true" background="#506dfb" showInline="true"/>
    <Label value="bridges" smart="true" background="#c42727" showInline="true"/>
    <Label value="buses" smart="true" background="#32ec79" showInline="true"/>
    <Label value="chimneys" smart="true" background="#109d6e" showInline="true"/>
    <Label value="crosswalks" smart="true" background="#90ce1c" showInline="true"/>
    <Label value="fire hydrants" smart="true" background="#0cc0ed" showInline="true"/>
    <Label value="motorcycles" smart="true" background="#000000" showInline="true"/>
    <Label value="parking meters" smart="true" background="#000000" showInline="true"/>
    <Label value="stairs" smart="true" background="#000000" showInline="true"/>
    <Label value="taxis" smart="true" background="#000000" showInline="true"/>
    <Label value="tractors" smart="true" background="#000000" showInline="true"/>
    <Label value="traffic lights" smart="true" background="#000000" showInline="true"/>
    <Label value="vehicles" smart="true" background="#000000" showInline="true"/>


    <Label value="Bus Eraser" smart="true" background="#000000" showInline="true"/>
  </KeyPointLabels>
</View>

[ODAncona]

The primary problem lies in exporting the COCO format, which requires the use of polygon labels. If you only have brush labels, this method will not function properly. I got a special case where I started manually with polygons and continued with the SAM ml backend integration. Therefore I've got two kinds of labels (brush and polygons) and the export breaks. It would be cool to have something working.

I found this online:

import json
import numpy as np
from pycocotools import mask
from skimage import measure

ground_truth_binary_mask = np.array([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  1,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0]], dtype=np.uint8)

fortran_ground_truth_binary_mask = np.asfortranarray(ground_truth_binary_mask)
encoded_ground_truth = mask.encode(fortran_ground_truth_binary_mask)
ground_truth_area = mask.area(encoded_ground_truth)
ground_truth_bounding_box = mask.toBbox(encoded_ground_truth)
contours = measure.find_contours(ground_truth_binary_mask, 0.5)

annotation = {
        "segmentation": [],
        "area": ground_truth_area.tolist(),
        "iscrowd": 0,
        "image_id": 123,
        "bbox": ground_truth_bounding_box.tolist(),
        "category_id": 1,
        "id": 1
    }

for contour in contours:
    contour = np.flip(contour, axis=1)
    segmentation = contour.ravel().tolist()
    annotation["segmentation"].append(segmentation)
    
print(json.dumps(annotation, indent=4))

[winter-fish]

I don't know if you solved it or not, I had the same problem :(

[ODAncona]

Thank you for sharing, I actually integrated SAM to labelstudio manually

[caitlinwheeless]

With 1.18, BrushLabels are supported for COCO export, and KeyPointLabels are coming soon. Thank you for opening this.

https://github.com/HumanSignal/label-studio/releases/tag/1.18.0