r/sharepoint • u/CheesyHamSammich • 22h ago

SharePoint 2019 Recurring CSV uploads to SharePoint keep becoming the fragile part of our SQL jobs

About two dozen CSV extracts end up in different SharePoint folders for departments around the company. The data comes from SQL Server, mostly on weekly schedules.

Generating the files is rarely the problem. Some come from SSIS, some from SQL Agent jobs, and then a few different PowerShell scripts handle the SharePoint upload.

That's the part I keep fixing.

Someone renames or moves a folder and the upload fails, while the SQL side of the job has already completed successfully. From the department's point of view, this week's file just never appeared.

The scripts have also accumulated over the years, so there isn't really one place to check what was delivered and what failed.

Has anyone standardised this kind of recurring file delivery into SharePoint? I'm less interested in rebuilding the SQL extracts than replacing the collection of upload scripts with something easier to monitor.

3 Upvotes
  • permalink
  • reddit

100% Upvoted

4 comments sorted by

3

u/ejaya2 20h ago

That’s a user access problem on the Sharepoint end, they have more rights than they should if they fail to understand that the folder structures have to remain as is for them to get their reports.

Lower their permission on the site or library to Read.

3

u/2much2want 20h ago

Why wouldn’t you just place the files directly into a document library without folders ? And start using meta data columns instead of folders

1

u/toddklindt MVP 15h ago

I assume you're using the PnP.PowerShell module in your script. Are you using Resolve-PnPFolder to see if the folder the file is supposed to go into exists? It checks to see if the folder is there and if it's not it creates it.

•

u/BraIVd 27m ago

The awkward part is that the file can be generated successfully while the actual delivery still fails. Those really need to be treated as two separate states.

With this many recurring uploads, keeping the delivery in one place would probably help more than touching the SQL jobs themselves. Skyvia can send SQL Server query results into SharePoint Files on a schedule and keep the run history together instead of spreading it across separate scripts.