Merge branch 'develop' into 'fb-optic-2152/zoom-presets-option-not-optimized-for-dark-mode'

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/14932391987

Author: ricardoantoniocm

docs/source/guide/helm_values.md

@@ -4,7 +4,7 @@ short: Available Helm values
 tier: all
 type: guide
 order: 72
-order_enterprise: 72
+order_enterprise: 74
 meta_title: Available Helm values for Label Studio Helm Chart
 meta_description: For cases when you want to customize your Label Studio Kubernetes deployment, review these available Helm values that you can set in your Helm chart.
 section: "Install & Setup"

docs/source/guide/install_k8s_airgapped.md

@@ -4,7 +4,7 @@ short: Airgapped Server
 tier: all
 type: guide
 order: 71
-order_enterprise: 71
+order_enterprise: 73
 meta_title: Install Label Studio without public internet access
 meta_description: Install Label Studio without public internet access to create machine learning and data science projects in an airgapped environment. 
 section: "Install & Setup"

docs/source/guide/install_prompts.md

@@ -0,0 +1,167 @@
+---
+title: Install Prompts in an on-prem environment (optional)
+short: Install Prompts
+type: guide
+tier: enterprise
+order: 0
+order_enterprise: 71
+meta_title: Install Prompts
+meta_description: Install Prompts in a Label Studio Enterprise on-prem environment
+section: "Install & Setup"
+parent: "install_k8s"
+parent_enterprise: "install_enterprise_k8s"
+---
+
+Installing Prompts in an on-prem environment requires installing Adala, our data labeling agent microservice.
+
+You only need to complete these steps if you want to use Prompts. For more information, see our [Prompts overview](prompts_overview).
+
+
+## Prerequisites
+
+- Kubernetes cluster **v1.24** or later
+- Helm **v3.8.0** or later
+- Docker CLI (for logging into Docker Hub)
+
+## Resource requirements
+
+Before installing, ensure your Kubernetes cluster can provide the following minimum resources for Adala:
+
+| Resource | Requirement |
+| --- | --- |
+| CPU | 6 cores |
+| Memory | 12 GB |
+
+## 1. Authenticate to Docker Hub and validate access
+
+You will need your Docker Hub username and password. If you do not have them, [request access from the HumanSignal team](mailto:[email protected]).
+
+Log in to DockerHub to access the private OCI repository:
+
+```bash
+
+docker login -u CUSTOMER_USERNAME
+```
+
+When prompted, enter your Docker Hub password.
+
+Then verify your credentials and access:
+
+```bash
+helm pull oci://registry-1.docker.io/heartexlabs/adala
+```
+
+Expected output:
+
+```bash
+Pulled: registry-1.docker.io/heartexlabs/adala:X.X.X
+Digest: sha256:***************************************************
+```
+
+## 2. Create a Kubernetes secret for image pulling
+
+Create a Kubernetes secret to allow your cluster to pull private Adala images:
+
+```bash
+kubectl create secret docker-registry heartex-pull-key \
+  --docker-server=https://index.docker.io/v2/ \
+  --docker-username=CUSTOMER_USERNAME \
+  --docker-password=CUSTOMER_PASSWORD
+```
+
+## 3. Prepare your custom values file
+
+Create a file named `custom.values.yaml` with the following contents:
+
+```yaml
+adala-app:
+  deployment:
+    image:
+      tag: 20250428.151611-master-592e818
+      pullSecrets:
+        - heartex-pull-key
+adala-worker:
+  deployment:
+    image:
+      tag: 20250428.151611-master-592e818
+      pullSecrets:
+        - heartex-pull-key
+```
+
+!!! note
+    Replace the `image.tag` with the appropriate version if necessary.
+
+
+## 4. Create a dedicated namespace for Adala
+
+Create a dedicated namespace `prompt` for Adala:
+
+```bash
+kubectl create namespace prompt
+```
+
+## 5. Install the Adala Helm chart
+
+Run the following command to install **Adala** using your custom values:
+
+```bash
+helm install lse oci://registry-1.docker.io/heartexlabs/adala --values custom.values.yaml
+```
+
+## 6. Validate that Adala is running
+
+Check if all pods in the `prompt` namespace are in the **Running** or **Completed** state:
+
+```bash
+kubectl get pods -n prompt
+```
+
+You should see output where all pods have `STATUS` set to `Running`, for example:
+
+```
+NAME                                  READY   STATUS    RESTARTS       AGE
+adala-adala-app-d4564ffd7-gtmhx       1/1     Running   0              100m
+adala-adala-kafka-controller-0        1/1     Running   0              110m
+adala-adala-kafka-controller-1        1/1     Running   0              111m
+adala-adala-kafka-controller-2        1/1     Running   0              113m
+adala-adala-redis-master-0            1/1     Running   0              125m
+adala-adala-worker-5d87f97f76-mq952   1/1     Running   0              111m
+
+```
+
+If any pod is not running, you can investigate further:
+
+```bash
+
+kubectl describe pod <pod-name> -n prompt
+```
+
+or
+
+```bash
+kubectl logs <pod-name> -n prompt
+```
+
+## 7. Update the Label Studio `values.yaml` file
+
+You will need to update the `global` section of your Label Studio Enterprise `values.yaml` file to include the following:
+
+* Add the Adala endpoint, which will allow Label Studio to connect to Adala. 
+* Add the Prompts feature flag, to enable Prompts visibility within Label Studio. 
+
+
+```yaml
+global:
+  extraEnvironmentVars:
+    PROMPTER_ADALA_URL: http://adala-adala-app.prompt:8000
+  featureFlags:
+    fflag_feat_all_dia_835_prompter_workflow_long: true
+```
+
+Note the following for `PROMPTER_ADALA_URL`:
+
+- `prompt` is the namespace where Adala is installed.
+- `adala-adala-app` is the name of the Adala service automatically created by the Helm release.
+- Port `8000` is the default port where Adala listens.
+
+After updating the values file, redeploy Label Studio to apply the changes.

docs/source/guide/prompts_overview.md

@@ -38,10 +38,13 @@ With Prompts, you can:
 | **Network access** | If you are using a firewall or restricting network access to your OpenAI models, you will need to allow the following IPs: <br>3.219.3.197 <br>34.237.73.3 <br>4.216.17.242 |
 | **Required permissions** | **Owners, Administrators, Managers** -- Can create Prompt models and update projects with auto-annotations. Managers can only apply models to projects in which they are already a member. <br><br>**Reviewers and Annotators** -- No access to the Prompts tool, but can see the predictions generated by the prompts from within the project (depending on your [project settings](project_settings_lse)).  |
 | **ML backend support** | Prompts should not be used with a project that is connected to an ML backend, as this can affect how certain evaluation metrics are calculated. |
-| **Enterprise vs. Open Source** | Label Studio Enterprise (Cloud only)<br />Starter Cloud|
+| **Enterprise vs. Open Source** | Label Studio Enterprise <br />Starter Cloud|
 
 </div>
 
+!!! note
+    For information on installing Prompts for on-prem environments, see [Install Prompts](install_prompts).
+
 ## Supported base models
 
 <div class="noheader rowheader">

label_studio/feature_flags.json

@@ -2364,6 +2364,33 @@
       "version": 4,
       "deleted": false
     },
+    "fflag_feat_front_leap_2036_annotations_summary": {
+      "key": "fflag_feat_front_leap_2036_annotations_summary",
+      "on": false,
+      "prerequisites": [],
+      "targets": [],
+      "contextTargets": [],
+      "rules": [],
+      "fallthrough": {
+        "variation": 0
+      },
+      "offVariation": 1,
+      "variations": [
+        true,
+        false
+      ],
+      "clientSideAvailability": {
+        "usingMobileKey": false,
+        "usingEnvironmentId": false
+      },
+      "clientSide": false,
+      "salt": "a0c67bbfb44743d6903c9445033c59c3",
+      "trackEvents": false,
+      "trackEventsFallthrough": false,
+      "debugEventsUntilDate": null,
+      "version": 2,
+      "deleted": false
+    },
     "fflag_feat_front_leap_482_self_serve_short": {
       "key": "fflag_feat_front_leap_482_self_serve_short",
       "on": false,
@@ -4054,13 +4081,13 @@
     },
     "fflag_fix_leap_2052_detect_empty_filters_at_next_task_endpoint_short": {
       "key": "fflag_fix_leap_2052_detect_empty_filters_at_next_task_endpoint_short",
-      "on": false,
+      "on": true,
       "prerequisites": [],
       "targets": [],
       "contextTargets": [],
       "rules": [],
       "fallthrough": {
-        "variation": 1
+        "variation": 0
       },
       "offVariation": 1,
       "variations": [
@@ -4076,7 +4103,7 @@
       "trackEvents": false,
       "trackEventsFallthrough": false,
       "debugEventsUntilDate": null,
-      "version": 2,
+      "version": 3,
       "deleted": false
     },
     "fflag_fix_leap_246_multi_object_hotkeys_160124_short": {

label_studio/io_storages/README.md

@@ -184,4 +184,29 @@ The Storage Proxy API behavior can be configured using the following environment
 | `RESOLVER_PROXY_MAX_RANGE_SIZE` | Maximum size in bytes for a single range request | 7*1024*1024 |
 | `RESOLVER_PROXY_CACHE_TIMEOUT` | Cache TTL in seconds for proxy responses | 3600 |
 
-These optimizations ensure that the Proxy API remains responsive and resource-efficient, even when handling large files or many concurrent requests.
+These optimizations ensure that the Proxy API remains responsive and resource-efficient, even when handling large files or many concurrent requests.
+
+## Multiple Storages and URL Resolving
+
+There are use cases where multiple storages can/must be used in a single project. This can cause some confusion as to which storage gets used when. Here are some common cases and how to set up mutliple storages properly.
+
+### Case 1 - Tasks Referencing Other Buckets
+* bucket-A containing JSON tasks
+* bucket-B containing images/text/other data
+* Tasks synced from bucket-A have references to data in bucket-B
+
+##### How To Setup
+* Add storage 1 for bucket-A
+* Add storage 2 for bucket-B (might be same or different credentials than bucket-A)
+* Sync storage 1
+* All references to data in bucket-B will be resolved using storage 2 automatically
+
+### Case 2 - Buckets with Different Credentials
+* bucket-A accessible by credentials 1
+* bucket-B accessible by credentials 2
+
+##### How To Setup
+* Add storage 1 for bucket-A with credentials 1
+* Add storage 2 for bucket-B with credentials 2
+* Sync both storages
+* The appropriate storage will be used to resolve urls/generate presigned URLs

label_studio/io_storages/base_models.py

@@ -27,7 +27,7 @@
 from django.utils import timezone
 from django.utils.translation import gettext_lazy as _
 from django_rq import job
-from io_storages.utils import get_uri_via_regex
+from io_storages.utils import get_uri_via_regex, parse_bucket_uri
 from rq.job import Job
 from tasks.models import Annotation, Task
 from tasks.serializers import AnnotationSerializer, PredictionSerializer
@@ -255,8 +255,19 @@ def can_resolve_scheme(self, url: Union[str, None]) -> bool:
             return False
         # TODO: Search for occurrences inside string, e.g. for cases like "gs://bucket/file.pdf" or "<embed src='gs://bucket/file.pdf'/>"
         _, prefix = get_uri_via_regex(url, prefixes=(self.url_scheme,))
-        if prefix == self.url_scheme:
-            return True
+        bucket_uri = parse_bucket_uri(url, self)
+
+        # If there is a prefix and the bucket matches the storage's bucket/container/path
+        if prefix == self.url_scheme and bucket_uri:
+            # bucket is used for s3 and gcs
+            if hasattr(self, 'bucket') and bucket_uri.bucket == self.bucket:
+                return True
+            # container is used for azure blob
+            if hasattr(self, 'container') and bucket_uri.bucket == self.container:
+                return True
+            # path is used for redis
+            if hasattr(self, 'path') and bucket_uri.bucket == self.path:
+                return True
         # if not found any occurrences - this Storage can't resolve url
         return False
 

label_studio/io_storages/functions.py

@@ -54,6 +54,5 @@ def get_storage_by_url(url: Union[str, List, Dict], storage_objects: Iterable[Im
         for storage_object in storage_objects:
             if storage_object.can_resolve_url(url):
                 # note: only first found storage_object will be used for link resolving
-                # probably we need to use more advanced can_resolve_url mechanics
-                # that takes into account not only prefixes, but bucket path too
+                # can_resolve_url now checks both the scheme and the bucket to ensure the correct storage is used
                 return storage_object

label_studio/io_storages/s3/serializers.py

@@ -68,6 +68,8 @@ def validate(self, data):
         except TypeError as e:
             logger.info(f'It seems access keys are incorrect: {e}', exc_info=True)
             raise ValidationError('It seems access keys are incorrect')
+        except KeyError:
+            raise ValidationError(f'{storage.url_scheme}://{storage.bucket}/{storage.prefix} not found.')
         return data
 
 

label_studio/organizations/api.py

@@ -117,8 +117,8 @@ class OrganizationMemberListAPI(generics.ListAPIView):
     pagination_class = OrganizationMemberListPagination
 
     def _get_created_projects_map(self):
-        members = self.get_queryset()
-        user_ids = members.values_list('user_id', flat=True)
+        members = self.paginate_queryset(self.filter_queryset(self.get_queryset()))
+        user_ids = [member.user_id for member in members]
         projects = (
             Project.objects.filter(created_by_id__in=user_ids, organization=self.request.user.active_organization)
             .values('created_by_id', 'id', 'title')
@@ -135,24 +135,29 @@ def _get_created_projects_map(self):
         return projects_map
 
     def _get_contributed_to_projects_map(self):
-        members = self.get_queryset()
-        user_ids = members.values_list('user_id', flat=True)
-        projects = (
-            Annotation.objects.filter(
-                completed_by__in=user_ids, project__organization=self.request.user.active_organization
-            )
-            .values('completed_by', 'project__id', 'project__title')
+        members = self.paginate_queryset(self.filter_queryset(self.get_queryset()))
+        user_ids = [member.user_id for member in members]
+        org_project_ids = Project.objects.filter(organization=self.request.user.active_organization).values_list(
+            'id', flat=True
+        )
+        annotations = (
+            Annotation.objects.filter(completed_by__in=list(user_ids), project__in=list(org_project_ids))
+            .values('completed_by', 'project_id')
             .distinct()
         )
-        projects_map = {}
-        for project in projects:
-            projects_map.setdefault(project['completed_by'], []).append(
+        project_ids = [annotation['project_id'] for annotation in annotations]
+        projects_map = Project.objects.in_bulk(id_list=project_ids, field_name='id')
+
+        contributed_to_projects_map = {}
+        for annotation in annotations:
+            project = projects_map[annotation['project_id']]
+            contributed_to_projects_map.setdefault(annotation['completed_by'], []).append(
                 {
-                    'id': project['project__id'],
-                    'title': project['project__title'],
+                    'id': project.id,
+                    'title': project.title,
                 }
             )
-        return projects_map
+        return contributed_to_projects_map
 
     def get_serializer_context(self):
         context = super().get_serializer_context()