(this post was written using a SharePoint 2010 environment)
So here’s the scenario, we have a document library that contains agendas, minutes, reports and other files related to meetings. The document library has a custom column called ‘Meeting Date’, a date column that is mandatory for all files that are uploaded.
We are required to create a view that is grouped by meeting date, sorted chronologically. Let’s give it a go!
Step one – prerequisites
My setup to achieve this was as follows:
A document library
An additional date column ‘Meeting Date’
Step two – configure the library
Navigate to the document library
Create a new calculated column called ‘Month of Meeting’
So this is a real world example of something I was asked to do recently. There was an existing a – z page which I was tasked with updating, so I’ve written about how I did it in both classic and modern SharePoint…
If you do want to extend this to have the drop-down item edit menu, just add ListItemMenu=”TRUE” to the field in SharePoint Designer. You can add this after the LinkToItem=”TRUE” and to multiple columns in the list view.
I’ve been suitably inspired by Andrew Warland’s fantastic two-part series documenting his approach and migration to SharePoint Online, so much so that I thought it would be a fun series to write about my own experiences.
It is’nt my intention to necessarily document Microsoft best practice in this series, rather just to explore some of the challenges, sucesses and experiences I notice along the way.
The current situation
My organisation has recently made the decision to move to to the cloud, with O365 being the naturally preferred destination. SharePoint has been well embedded, and heavily used within the business for several years, with on-premises SharePoint 2010 currently in production.
Finally, in terms of the SharePoint architecture and data volume, there are only three web applications to merge together as part of the migration effort. However, there are several site collections within our main intranet web app, plus many sub-sites nested within them, meaning the huge database sizes behind these site collections could prove difficult come migration time.
A note on the new, flat structure
Our current environment has a well established top-down structure in place that is generally consitent across the environment.
Having already made the investment in ShareGate, this will be the tool of choice for the migration. In the version 11.0 release of ShareGate, a new restructure option now allows you to promote sub-sites to top-level sites post inital migration from the source SharePoint environment.
Considerations for a successful migration plan
One of the biggest issues to be resolved before we can start any sort of migration activity, is the fact that we have several content databases well over the 200GB recommended general use size limit.
Microsoft best practice suggests that any environment that has site collections, sites, content databases, libraries or lists that exceed the software boundaries and limits should be remediated prior to any migration activity. In this case, the main idea is to split each content database that exceeds 200GB into seperate content db’s, and where neccessary, move or promote sub-sites to site collections and attach new db’s.
Armed with the knowledge of the recent restrcuture functionality coming to ShareGate, plus my own personal feeling that any remediation activities to our current environment may in of itself carry adverse risk to the estate we proposed a different approach.
With all the reporting capabilities at our disposal via ShareGate, I was able to get a firm grasp of what resides within each site collection in our environment, in terms of:
The size of each sub-site underneath the top-level
Number/ size of libraries and lists
Number of items in each of the above
Any workflows running in any of the above
From this I ran a trial migration of a sub-site from SharePoint 2010 to a newly created team site in SharePoint Online.
Before I kicked off the migration, I ran the source analysis tool within the Migration > Plan section of ShareGate. I noted the following obersavations:
The source analysis within “migration” in the ShareGate tool, although listed as only being able to analyze up to SharePoint 2013, does in-fact work for 2010
The source analysis cannot run at the sub-site level, meaning that you need to run it at the site collection level then just filter down to the sub-site in question through the report itself
Source analysis gives you a report of all checked-out files within a source site.From this, I created a simple view within each of the libraries that contained checked-out files to send to the site owners for action
The trial migration completed successfully as expected, however there were several interesting results I noted:
1. Everyone receieves a welcome email
If you migrate the permissions, once the source permission groups migrate each user will recieve a welcome email to the new SharePoint Online site.
Publishing sites seem to be the trickiest to migrate, especially those with custom master pages or page layouts. When migrating publishing sites, the Pages library is migrated wholesail, meaning the content won’t reside in the SitePages library (where new client-side pages are located).
3. Un-editable modern homepage
After the migration had completed, the new team site homepage threw up an error every time you tried to edit it.
I tried some of the documented resolution steps found here, but none of them worked for me. My solution was to just create a new page to replace the broken homepage, add all the relevant webparts and make this one the new default homepage.
Transforming classic publishing site pages to client-side pages
Publishing site pages will all be migrated as classic SharePoint pages, without the modern look and feel of a client-side page. My understanding is that for publishing pages with custom page layouts, additional metadata or custom content types will need to be transformed via PowerShell and creating a custom mapping file.
(I’m planning on writing a seperate blog post walking through an advanced publishing page transformation in the near future)
Its also worth considering that in the release notes for ShareGate 11.0 it makes mention of the fact they are researching the ability to transform classic to modern pages, so that could well simplify this process in a future release.
Overall, I was happy with our trial migration and believe it is a viable approach for us to move from on-prem to O365. Some lessons learned for myself would be to consider and SharePoint permissions audit prior to migration to remove any unecessary permissions, send an inventory out to site owners aswell as checked-out files, all in the name of reducing the migration effort.
This will be an ongoing series of posts, which i’ll focus more the on the nitty-gritty of the migration effort than anything else, but as always if there is any feedback or suggestions on how to improve this site, please let me know!
Add the following field reference in between the opening <ViewFields> and closing </ViewFields>
Add the field reference in the display order you would like it to appear in the view
Press the Save icon to save your changes
Press the Preview button to see your view in action in the browser
Now you will notice there is a new column being displayed “Path”, that is showing us the full location of the file or folder in the libary. You’ll also notice that this path will display data when at the library root, or in any folders or sub-folders in the library.
Taking this one step further, what if we wanted to show files of a certain type, then create a view that groups these files by their folder location? Guess what, that’s exactly what I did!
Navigate to your library > create a new view as before, this time base your new view off the one you just created
If you wish to only show files of a particular type, use the filter by settings (for example below is filtered to only shows Word documents)
Make sure “show all items without folders” is selected
Back in SharePoint Designer > Open up the view you just created
Scroll down until you see the opening <Query> tag and add the following beneath it:
With Microsoft retiring SharePoint 2010 designer workflows, plus the movement away from SharePoint Designer in general, a few readers have asked for a solution that works with modern SharePoint.
When researching this I considered whether suggesting to use SharePoint Designer 2013, as the above solution would still work in SharePoint Online using SPD 2013. But, as Microsoft say themselves although SPD 2013 remains supported, it’s depreciated – so I decided to go in a different direction.
The below example walks you through how you can create a flow in Power Automate to update a file after it’s been created to have the folder path shown in the document library view:
For this solution you will need to have access to create Flows in Power Automate, as well as an existing Document Library created in SharePoint Online:
Navigate to the document library you wish to show the folder path for
Add a single line of text column to the document library > give it a name (I called mine FolderPath)
Under the ellipsis, press Automate > Power Automate > Create a flow
In Power Automate, either use an existing, relevant template or start from blank
The trigger action should be When a file is created (properties only)
Set the Site Address and Library Name where you want to add the folder path
Insert a new step > select Update file properties.
Set the following values for the update file properties step:
Site Address: same as previous step
Library Name: same as previous step
FolderPath: Folder path
NOTE: The FolderPath within the Update file properties step is the custom column we created earlier. The Folder path (highlighted in red) is dynamic content available within the step in the flow. The folder path dynamic content is the path to the folder the item is in, relative to the site address.
Here’s the flow in it’s entirety:
At this point test and save your flow to make sure it is working as expected 🙂
Bonus #1 – turn your folder path column into a hyperlink column
So if like me you want to take this one step further, wouldn’t it be good if we could easily make our newly showing folder paths, actual hyperlinks to the folders? Well the good news is you can!
Navigate back to your document library > click on the FolderPath column > Format this column
Under Apply formatting to make sure FolderPath is selected
Paste the following JSON into the custom formatting box:
NOTE: for more information on turning field values into hyperlinks, check out this awesome sample from sp-dev-list-formatting.
Your FolderPath column values should now be legitimate hyperlinks that click through to the relevant folders
Bonus #2 – update existing files in the document library
This was another suggestion from a reader with regards to how to update files that existed in the document library before the flow was created.
Running a flow manually for individual files
When I began to consider how to do this I started by looking at ways to manually start the flow.
It appears the only real way to do this is to create a new column that adds a button next to each file, that allows you to run the flow. I’m not really enamoured by this approach as it doesn’t seem ideal to have an extra column to run a flow showing on every file in your library. If this is something you would like to pursue then I would recommend this great article by WonderLaura who has the process of creating a button to trigger a flow covered!
Update our flow to update all files if folder path is empty
My solution to this problem was to update the flow we created earlier to get the properties for all files in the library, then add a condition that checks if the FolderPath column is empty, then if yes runs our flow as before.
First, I added a Get files (properties only) action which gets all the files from the source library
Then I added a new Condition action, which simply checks if the FolderPath column we created is equal to null. You will also notice a new Apply to each action will be created
I then moved the previous Apply to each action into the “If yes” condition
Data retention and deletion…I’m sure this is a something that anyone involved in Office 365, SharePoint on information management in general gets fed up of saying since the recent GDPR legislation!
Recently we have been rationalising and cleaning up our data in preparation for moving to Office 365. We are starting with SharePoint as the first target repository or silo of content.
The general consensus is to delete files and folders over 7 years old unless there is a pre-existing data retention policy to adhere to. So the next task is to identify those files that fall within our threshold, and ultimately delete.
Luckily, we have Tree Size Pro and ShareGate so I was able to relatively easily identify the files in question (there were a lot!).
As our SharePoint environment is a) rather full; and b) rather old, I made the decision to incrementally delete files rather than en-masse to mitigate risk, targeting the lists/libraries containing the most out of date content. I started by creating a view in the first library – library A with the following parameters:
Standard library view
Filtered by Created Date if less than or equal to 01/01/2011
Folders or Flat: Show items inside folders Show this view: In all folders
(all other settings are left default)
Results this returned looked good, I could see folders and files in this view that matched the criteria – brilliant! Based on my previous statement I decided to delete in batches out of working hours, again to mitigate risk. I deleted first from library A, then from the first stage and finally from the second stage recycle bin all in this fashion.
I had permanently deleted around 50% of the total volume of content to be deleted from library A when we started to receive reports of current files being ‘missing’ from library A…not a good day.
After these reports were investigated they were indeed true. It turns out that when folders are included within a library view, folders that match the filter will be shown in the view, regardless of whether the files inside match.
We tested the view exluding folders and all the files returned matched the filter criteria. The same results were demonstrated from a SharGate report of the same nature. The report of all files over 7 years old brought back folders over 7 years old, but they also contained files that were newer.
At present, we are not entirely sure as to why these filters are not able to drill down past a top-level folder. It appears to be difficult to specify via view settings to only show files within folders, including the folder itself that matches the criteria.
We have decided to omitt folders from our reports and views going forward and to solely focus on files as this is the most reliable way we can delete files.
Bonus: for those of you with ShareGate, heres an example of my report we created to bring back all files over 7 years old, excluding folders. I ran this report across the entire intranet application over a weekend and it worked a treat 🙂
(This post was written using a SharePoint 2010 environment)
So you’ve got a SharePoint site, it all looks good (well, as good as it can!) but you notice that the Publish tab isn’t available in the ribbon.
First things first you check the site settings to see if SharePoint Server Publishing is turned on.
If you get to this point and your still no further forward it’s likely that your site wasn’t set up as a publishing site, but if you follow the steps below and your pages will be able to be published in no time.
Open the site in question, then go to Site Actions > View All Site Content
Open the SitePages library
Under Library Tools > Library, select Library Settings
Under General Settings > Versioning Settings, turn on Create major and minor (draft) versions > press OK
Go back to the original page, you will now see the publishing tab has appeared!
Recently, the same .NET security update was applied to our SharePoint 2010 farm, which in turn caused the failing on start error to present itself again across all the workflows in the farm.
After identifing the issue soon after the update was applied, we decided to follow the same tact as before and roll back the patches, restart the servers and re-test the workflows – However, this time the results were different.
What was different?
Previously, rolling back the security update and any other patches added during this time, plus restarting the servers “fixed” the issue. This time, the same process did not yeald the same results and the workflows were still broken.
After performing the steps above, we observed that standard SharePoint workflows with a pause started to run sucessfully again, but Nintex workflows with a pause step either failed on start, or completed but errored after the pause step and sent an error notification.
How we fixed it…
So this time we followed the updated step-by-step guide provided below on how to update the web.config files and OWS timer files via Add-CodeDomAuthorizedTypeToOWSTimerConfig.ps1 PowerShell script on the SharePoint Application server.
We ran the script as recommended, which re-added the assemblies and dependancies to the OWSTimer config file and the web.config files on associated web servers and this in fact fixed the issue! As the script does an IIS reset/ Timer Job recycle we didnt even need to restart the servers!
I hope the that tidbit regarding the nintex workflow pauses helps someone else 🙂