Could really use a log

I’ve got a flow with a subflow and then a subflow after that.

When the inner-most flow fails, which is will do if it can’t find a product name in a Google Sheet, it would be useful if I could output a log message and have it continue.

When everything is complete, I would have a list of all the failures, which I can then correct and run the flow again.

Currently, when the inner-most flow fails, I have to examine that run, fix the error and start over again. This isn’t practical for the number of missing data items there are in my sheet.

Would it be possible to be able to add logging?

Makes sense — we could definitely explore that, though I think it would require a bit of scoping to figure out exactly how it would work. You can still do this within a workflow.

Since your problem statement is that when a subflow fails you want to add a log message and continue the flow, you can wrap the subflow in an error shield and enable “pass inputs through.” That way, you’ll have two different branches: one if the subflow works and produces the output, and another if it fails.

You can connect the error output to something like a find-and-replace node to append the log message and continue with the rest of the flow.

Here’s an example setup: https://www.gumloop.com/pipeline?workbook_id=4vVYym4kyxm1Tfg8HR1fJt&run_id=X2jmx5z39GDVSEcrNPaYEF

Let me know if this makes sense and works for you.

That might work, I’ll give it a whirl. Thanks.

I wasn’t able to get it working. I wrapped the Google Reader node with an error shield and sent the error to an Output node. The Error Shield is in the subflow “Process Single Product.”

With this setup, where will I see the errors? Perhaps I should write those to a Google doc?

https://www.gumloop.com/pipeline?workbook_id=2NtQrK3Y26BWoxehX5bbfm

If I’m understanding this correctly, the error will show in the run log for each subflow run. But how do I know which run has an error? And I have three levels of a hierarchy here…that’s a lot of clicking just to find errors.

I don’t see an Error Shield anywhere in this flow, perhaps it was deleted?

Regardless just to clarify the Error Shield approach, it does not print or output the error that occured, it simply catches the error and protects the node from failing. When you enable Pass Inputs Through, you then have two branches:

  • One with the normal outputs, denoted by the white circle at the bottom of the node/subflow
  • One with the inputs if the node/subflow errors out, denoted by the circle with the orange output at the bottom of the node

This branching allows you to treat the inputs that error’d out separately (again it only outputs the inputs that ran into an error, the actual error message or log is not available).

Eg:

One way to perhaps Log errors is to output the inputs which ran into an error along with their run links in a Doc/Spreadsheet. Eg: https://www.gumloop.com/pipeline?workbook_id=3biZtyf4Nuz3LCjhjCQrMn

My previous implementation here was to show how to proceed with the flow even when a node/subflow errors out, so in this case the subflow ran into an error since the search value was not present in sheet, hence we had the input available via the Error output which I converted to a standard error message using the Find & Replace node to continue moving along the flow.

I do understand your point regarding printing error messages directly as part of logging, will add that to the roadmap.

Sorry I pasted the wrong link. Here it is:
https://www.gumloop.com/pipeline?workbook_id=c3WQ32T8ADVeFiv3rnjVty

But I think we’re on the same page. The error is “trapped” inside of the run log and one needs to visit each run/node combination to collect them (unworkable) all unless they are sent to a doc.

I’ll try that now.

BTW, I just added Winston to one of my products and it works well:

Things are working now. The key is to use a Google doc to record the errors. It gives me a list of the products that the flow fails to find, I then paste that list into the spreadsheet, run the flow again and it’s clean. I’m working through the product names list now in groups of 20.

Two Random Bugs
In this work session, experienced two bugs.

  1. The initial Google Sheet Reader lost the link to the sheet after I changed the permissions of the sheet. Had to re-paste it. It seems that changing permissions shouldn’t have that side effect.
  2. The Google Reader placed a value in the inspector field on its own. I had to delete it to get the flow working again. This has happened twice to me.

Cheers,
André

Nice, glad you got it working!

  1. The initial Google Sheet Reader lost the link to the sheet after I changed the permissions of the sheet. Had to re-paste it. It seems that changing permissions shouldn’t have that side effect.

We actually have to enforce that to protect data.

  1. The Google Reader placed a value in the inspector field on its own. I had to delete it to get the flow working again. This has happened twice to me.

I see. I’m able to reproduce. This may have happened if you enabled the Search Value as a dynamic input and then disconnected it. Created a ticket to fix this. Appreciate you flagging this!

Also, sent over some credits for the great feedback and bug finds recently :slight_smile:

“We actually have to enforce that to protect data.”

That’s fair…but then shouldn’t you post an alert to the user that they have to reconnect a node rather than having them run the flow and wonder why it broke?

1 Like

Thanks. I appreciate the appreciation!

1 Like

This topic was automatically closed 4 days after the last reply. New replies are no longer allowed.