This week I’ve been working on replacing an ancient old Jitterbit install which upserts some CSV files to staging tables in Salesforce. Sounds easy and a good test for Logic Apps to see if it’s a viable choice for our integrations going forward. Of course it’s not as simple as you might think.
Problem 1
Logic apps doesn’t do CSV to JSON import naively.
Was a bit surprised to find this out and unfortunately our “cutting edge” German ERP pretty much only integrates via CSVs!
So the first job was to create a simple WebAPI in .Net Core 2.2 to accept a POST CSV payload (with headers) and send back the file in JSON. Be warned there is a bit of mucking about to get a WebAPI to accept “text/plain” content, have a look here to find out how to write a custom InputFormatter.
[HttpPost]
[Consumes("text/plain")]
[Produces("application/json")]
public ActionResult Post([FromBody] string body, [FromQuery] string delimiter = ",")
{
var resultSet = new JArray();
string[] csvLines = body.Split(Environment.NewLine);
var headers = csvLines[0].Split(delimiter).ToList();
foreach (var line in csvLines.Skip(1).Where(s => !string.IsNullOrWhiteSpace(s)))
{
var lineObject = new JObject();
var lineAttr = line.Split(delimiter);
for (int x = 0; x < headers.Count; x++)
{
lineObject[headers[x]] = lineAttr[x];
}
resultSet.Add(lineObject);
}
return resultSet;
}
At this point it was all working and looked like this:
Problem 2
You can only call the Salesforce API 1,000 time within 24 hours.
So I had all the file parsing working and added a Foreach loop to iterate over the array of JSON objects returned by the WebAPI, it was all working swimmingly until I started getting “TotalRequests Limit exceeded” error returning from the Salesforce connector.
Turns out you need to use the CreateJob connector call to do bulk data operations, I’ll detail this in part 2… once my Salesforce account unblocks due to too many API calls 🙂
Hi, Haven’t you created the second part of this blog? I am trying to do something similar, it would be helpful for me as well.