[Stop chat completions]

[AndraxDev]

I have the following code:

val completions: Flow<ChatCompletionChunk> = ai!!.chatCompletions(chatCompletionRequest)

completions.collect { v ->
    run {
        if (stopper) {
            stopper = false
            return@collect
        }
        if (v.choices[0].delta.content != "null") {
            response += v.choices[0].delta.content
            if (response != "null") {
                messages[messages.size - 1]["message"] = response
                adapter?.notifyDataSetChanged()
                saveSettings()
            }
        }
    }
}

I want to stop completions when user clicked a button. I tried to do this using stopper variable but id doesn't work. It there any official API solution to do this?

[AndraxDev]

UPD: Tried to do this by cancelling coroutine but it still not work

[AndraxDev]

As for now I found one solution - add button click listener directly to the coroutine scope:

CoroutineScope(...).launch {
    button.setOnClickListener {
        cancel();
    }
    
    try {
        // Generate response
    } catch (e: CancellationException) {
        // Stop generation, restore UI state
    }
}

But I think it's a temporary solution. Is there a better way to do this?

[aallam]

I would suggest something like:

val job = launch {
    try {
        openAI.chatCompletions(request).cancellable().collect()
    } catch (e: CancellationException) {
        println("Flow was cancelled as expected.")
    } catch (e: Exception) {
        error("Flow threw an unexpected exception: ${e.message}")
    }
}

button.setOnClickListener {
    job.cancel()
}

NB: cancellable() won't be required in the next release, cancellation should be handled much better.

[AndraxDev]

Thanks a lot! I have already implemented this feature. The only one thing that I must set listener directly inside launch {} because listener outside does not stop the job properly. UI state is restored but HTTP request is not cut and chat competion continues. It's just specified topology of my app.

[aallam]

listener outside does not stop the job properly. UI state is restored but HTTP request is not cut and chat competion continues

Indeed, this is exactly the part that should be fixed in the next release :)