Processing Posted File and Invoking OpenAI Assistant

[Bower]

Hi All,

These issues are not urgent!

I’ve encountered 2 issues described below.

• Processing a posted audio file via a webhook
• Calling an OpenAI Assistant that has a File Search tool

Audio File Post:
Trying to process an audio file posted via webhook.
Whether I use the Speech to text node or Google Save File node, the processing of the post seems to fail.
https://www.gumloop.com/pipeline?run_id=8fZRzVWRT2MeGZmDs58f9N&workbook_id=m2VnABK3oq83QS7QGaSCRz
Speech to Text Failed!
There was an error in your Speech to Text node. We can help you troubleshoot in the Gumloop Forum!

Other notes: When troubleshooting, I tried saving the posted file to google drive and the save file node failed in the same manner as the speech to text. Therefore, I think the issue is in reading the file from the web hook and not the nodes(?) Also, I saved the client file to disk then uploaded that to gum loop manually and was able to successfully convert the audio file to text so it is not a file issue.
For the client, I am using a simple python streamlit app:

# Read the saved file and encode it in base64
        with open(filename, "rb") as f:
            file_data = f.read()
            base64_file = base64.b64encode(file_data).decode('utf-8')
  
        payload = {
            "user_id": "Hsi1cuMynhMWoTbb9X2fqUoX1r23",
            "saved_item_id": "5wmksQLfoUv9eVDrKKhtZu",
            "pipeline_inputs": [{"input_name":"File", "value": base64_file}]
        }

If you need anything to help troubleshoot, please let me know!

Issue #2:

Symptom: Calling an OpenAI Assistant does not generate the expected response.
I have a simple OpenAI Assistant that uses the File Search tool. The File Search has an OpenAI vector store with some documents to be queried.

Issue: when I call this OpenAI Assistant via Gumloop, the assistant is invoked successfully, however, the response indicates that the assistant did not use the File Search tool. I can use the assistant in the OpenAI Playground successfully (returning results from the vector store).

https://www.gumloop.com/pipeline?workbook_id=dEZwc7nspSkyHySY93izoZ&run_id=UDz9zipe7wGrLnHE7dLdpC

Here is the top of Gumloop’s response from the assistant:

“I seem to have run into a bit of trouble accessing the files containing the information, but worry not! Here’s how you can find out the winter break schedule:” - the response then goes on with a generic model response without any vector store info.

If you need any additional help/info to troubleshoot, please let me know!

Thank you very much for your help.
Bower

[Gumloop_Bot]

Hey @Bower! If you’re reporting an issue with a flow or an error in a run, please include the run link and make sure it’s shareable so we can take a look.

1. Find your run link on the history page. Format: https://www.gumloop.com/pipeline?run_id={your_run_id}&workbook_id={workbook_id}

2. Make it shareable by clicking “Share” → ‘Anyone with the link can view’ in the top-left corner of the flow screen.
   

3. Provide details about the issue—more context helps us troubleshoot faster.

You can find your run history here: https://www.gumloop.com/history

[Wasay-Gumloop]

Hey @Bower - Thank you for the detailed information!

Issue #1

The Speech to Text node requires a valid media file input, the input from your run does not seems to be a valid file object. To clarify, when sending files via webhook they must exist on the Gumloop working directory: https://docs.gumloop.com/api-reference/file-operations/upload-file

An easier option would be to send the media file link and then enable Use Link in the node:

Issue #2

I’m not sure why that is to be honest. The node just links your OpenAI API key and the assistant, nothing else going on there. Is it the same if you test the assistant in a different environment or let’s say using the run code or custom node feature?

[Bower]

Hi Wasay,

Makes perfect sense now with the file upload - I got it working by posting the file to the workspace. I assume to use the By URL, one needs to upload the file to another media server then reference it? What are the limitations on workspace files and can you delete them programatically? For all intents and purposes everything is working, just side questions!

I’ll fiddle with the OpenAI Assistant. Thanks for your help, Wasay!

Bower

[Wasay-Gumloop]

Awesome!

• The limit to file uploads is 200MB
• If you use the URL option then yes you’ll need to upload the file to another service like Amazon S3 and then send the file URL. This is helpful if you want to store the files on your own database instead of the working directory
• You cannot delete files programmatically right now but it can be done manually: Working With Files - Gumloop

[Bower]

Hi Wasay, This is not urgent but I continue to troubleshoot the OpenAI Assistant. I created a simple assistant with code interpreter, uploaded a simple csv to analyze, and I get just generic results with using the OpenAI Assistant node:

https://www.gumloop.com/pipeline?workbook_id=hycdQrmdSCSQQa2fKPJ4Ti&run_id=YBpUb4kAEpkpFYa5bfREZa

When I use that assistant in the OpenAI Playground, it properly calls the code interpreter and responds.

I also call the agent from a Flowise node and it works.

It’s very odd - both the file search and the code interpreter (the only 2 tools I’ve tried) do not get invoked when I call the assistant here in gumloop.

Again, this isn’t a showstopper for me right now but any pointers appreciated. Granted, looks like this OpenAI API is changing soon so…shrug…

If you need any videos or info on how this is setup in OpenAI, please let me know.

Thanks!
Bower

[Wasay-Gumloop]

Hey @Bower - Thank you for testing and pointing this out. Could you try the custom OpenAI Assistant node here please: https://www.gumloop.com/pipeline?workbook_id=pqgyz2YXv2L1dq15RaeCY1

I also shared it with your account so you can look it up and add it from the node library.

Let me know if this outputs the expected response.

[Bower]

Hi Wasay, this works great! I got the proper output response as well as citations. Thanks very much! Please keep me posted and if there is anything else I can help to try/verify.

Thanks!
Bower

[Wasay-Gumloop]

Awesome, you can use the custom node for your workflows and it should work as expected.

[Bower]

Hi Wasay, Thank you again. I’m curious is it something to do with my assistants or is there an issue? Just want to understand if it is something I need to fix. Am I right, though, that this assistants interface is overhauled with the new Assistants API? Thanks for any context and your plans to support assistants going forward. Thanks!
Bower

[Wasay-Gumloop]

No issue with your assistant – we might need to upgrade our native assistant node to look into this edge case and fetch all tools. In the meantime to unblock you I created a custom node (its a feature on Gumloop, you can create on yourself).

You can use the custom node I created normally within any workflow or create a custom node yourself. Once we’re able to upgrade the native node I can send an update here.

Doc: https://docs.gumloop.com/nodes/custom_node_details
Video Tutorial: https://www.youtube.com/watch?v=yHjxbmdg-cI&ab_channel=Gumloop
Custom Node Workshop: https://www.youtube.com/watch?v=ovCCWfgWv4M

[Bower]

That custom node builder is so cool, I will try it out asap. I got the call api node up and running earlier today which is powerful. Thanks for your feedback! You can close this one out and I appreciate all the help!
Bower

[Wasay-Gumloop]

Awesome, thank you!!