Problems creating list or library views based on created date

The situation

Data retention and deletion…I’m sure this is a something that anyone involved in Office 365, SharePoint on information management in general gets fed up of saying since the recent GDPR legislation!

Recently we have been rationalising and cleaning up our data in preparation for moving to Office 365. We are starting with SharePoint as the first target repository or silo of content.

The general consensus is to delete files and folders over 7 years old unless there is a pre-existing data retention policy to adhere to. So the next task is to identify those files that fall within our threshold, and ultimately delete.

Luckily, we have Tree Size Pro and ShareGate so I was able to relatively easily identify the files in question (there were a lot!).

The setup

As our SharePoint environment is a) rather full; and b) rather old, I made the decision to incrementally delete files rather than en-masse to mitigate risk, targeting the lists/libraries containing the most out of date content. I started by creating a view in the first library – library A with the following parameters:

  • Standard library view
  • Filtered by Created Date if less than or equal to 01/01/2011
  • Folders or Flat: Show items inside folders
    Show this view: In all folders

(all other settings are left default)

Results this returned looked good, I could see folders and files in this view that matched the criteria – brilliant! Based on my previous statement I decided to delete in batches out of working hours, again to mitigate risk. I deleted first from library A, then from the first stage and finally from the second stage recycle bin all in this fashion.

The problem

I had permanently deleted around 50% of the total volume of content to be deleted from library A when we started to receive reports of current files being ‘missing’ from library A…not a good day.

After these reports were investigated they were indeed true. It turns out that when folders are included within a library view, folders that match the filter will be shown in the view, regardless of whether the files inside match.

We tested the view exluding folders and all the files returned matched the filter criteria. The same results were demonstrated from a SharGate report of the same nature. The report of all files over 7 years old brought back folders over 7 years old, but they also contained files that were newer.

Conclusion

At present, we are not entirely sure as to why these filters are not able to drill down past a top-level folder. It appears to be difficult to specify via view settings to only show files within folders, including the folder itself that matches the criteria.

We have decided to omitt folders from our reports and views going forward and to solely focus on files as this is the most reliable way we can delete files.

Bonus: for those of you with ShareGate, heres an example of my report we created to bring back all files over 7 years old, excluding folders. I ran this report across the entire intranet application over a weekend and it worked a treat 🙂

SG-report

File naming restrictions in SharePoint

Firstly, full credit for this information goes to Mischa Oudhof and this sysadmin blog for this really useful post in 2010! I thought i’d write it up here just for my own sanity (and incase that page were to ever not exist) plus add a few extra bits of my own.

So recently a user was trying to upload multiple documents into SharePoint 2010 by copying/pasting using file explorer for the destination library.  When copying across the files they received something like this message:

filename-error

What are the restrictions?

There are a whole load of restrictions when it comes to file name, length, characters within SharePoint. This affects site collections, sub-sites libraries or lists. Here are some of the more general restrictions:

  • File and folder name lengths can’t exceed 128 characters in both WSS 2.0 as WSS 3.0
  • Link list items can’t exceed 256 characters
  • The entire path of files can’t exceed 260 characters

Here are some, more specific restrcitions again applying to site collections, sub-sites, libraries or lists:

  • Can’t be longer than 128 characters
  • Can’t use: ~ # % & * { } \ : < > ? / + | ”
  • Can’t use the period character consecutively in the middle of a file name (blah…blah.docx)
  • Can’t use the period character at the end of a file name
  • Can’t use the period character at the start of a file name
  • Can’t end with:

    .files

    _files

    -Dateien

    _fichiers

    _bestanden

    _file

    _archivos

    -filer

    _tiedostot

    _pliki

    _soubory

    _elemei

    _ficheiros

    _arquivos

    _dosyalar

    _datoteke

    _fitxers

    _failid

    _fails

    _bylos

    _fajlovi

A deep dive into all the potential restrcitions can be found in the msdn blog here.

Things have changed in Office 365…

Microsoft announced in 2017 that the old maxpath limit has been expanded to 400 Unicode code units. Microsoft also announced more URL improvements for the support of # and % characters.

New SharePoint team site

References

http://www.sysadminsblog.com/microsoft/file-name-length-and-character-restrictions-for-sharepoint/  – Original write up by Mischa Oudhof of Sysadminblog.com

SharePoint and Nintex workflows failing on start after .NET security update

Updated


I’ve wrote part two on this issue with my full resolution steps here:

SharePoint and Nintex workflows failing on start – part two


The problem


I had this issue myself in the last week where EVERY SINGLE workflow across the farm on premise stopped working. SharePoint Designer and Nintex workflows all reported “Failed to start” when triggered to run.

The workflows stopped working due to a series of .NET security updates Microsoft released in September 2018. Microsoft released a public KB article on this – with resolution steps which can be found below:

But also this msdn blog post contains all the solution scripts and steps that includes Nintex workflows also (transcript below):

I noticed shortly after the fix was implemented that some of my SharePoint designer workflows were exhibiting odd behaviour. For example the screenshot below shows a SharePoint desinger workflow that previously worked without issue or errors in the history after the fix was applied:

Someone on reddit had already spotted this which drew my attention to the common issue, this only presents itself for workflows with pause steps!

I will update this post with my findings once this latest fix is applied.

Symptom

After applying .NET Security Only patch to resolve CVE-2018-8421 (Remote Code Execution Vulnerability) , all SharePoint out of the box Workflows fail to execute and the log will show an error like this:

09/13/2018 01:59:07.57 w3wp.exe (0x1868) 0x22FC SharePoint Foundation Workflow Infrastructure 72fs Unexpected RunWorkflow: Microsoft.SharePoint.SPException: <Error><CompilerError Line=”-1″ Column=”-1″ Text=”Type System.CodeDom.CodeBinaryOperatorExpression is not marked as authorized in the application configuration file.” /><CompilerError Line=”-1″ Column=”-1″ Text=”Type System.CodeDom.CodeBinaryOperatorExpression is not marked as authorized in the application configuration file.” /><CompilerError Line=”-1″ Column=”-1″ Text=”Type System.CodeDom.CodeBinaryOperatorExpression is not marked as authorized in the application configuration file.” /><CompilerError Line=”-1″ Column=”-1″ Text=”Type System.CodeDom.CodeBinaryOperatorExpression is not marked as authorized in the application configuration file.” /><CompilerError Line=”-1″ Column=”-1″ Text=”Type System.CodeDom.CodeBinaryOperatorExpression is not marked as authorized in the application configuration file.” /><CompilerError Line=”-1″ Column=”-1″ Text=”Type System.CodeDom.CodeBinaryOperatorExpression is not marked as authorized in the application configuration file.” /><CompilerError Line=”-1″ Column=”-1″ Text=”Type System.CodeDom.CodeBinaryOperatorExpression is not marked as authorized in the application configuration file.” /><CompilerError Line=”-1″ Column=”-1″…

The error suggest that System.CodeDom.CodeBinaryOperatorExpression is not in the authorized types.

Cause

Workflow Foundation (WF) will only run workflows when all the dependent types and assemblies are authorized in the .NET config file (or added explicitly via code) under this tree:

<configuration>

<System.Workflow.ComponentModel.WorkflowCompiler>

<authorizedTypes>

<targetFx>

However, after the update, the following lines are necessary for SharePoint 2013 and beyond:

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeBinaryOperatorExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodePrimitiveExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeMethodInvokeExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeMethodReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeFieldReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeThisReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodePropertyReferenceExpression” Authorized=”True” />

And for SharePoint 2007 and 2010, use these lines:

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeBinaryOperatorExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodePrimitiveExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeMethodInvokeExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeMethodReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeFieldReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeThisReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodePropertyReferenceExpression” Authorized=”True” />

Solution

The solution is to add explicitly the types to all web applications’ web.config:

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeBinaryOperatorExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodePrimitiveExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeMethodInvokeExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeMethodReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeFieldReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeThisReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodePropertyReferenceExpression” Authorized=”True” />

Or (for SharePoint 2007 and 2010):

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeBinaryOperatorExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodePrimitiveExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeMethodInvokeExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeMethodReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeFieldReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeThisReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodePropertyReferenceExpression” Authorized=”True” />

Please notice that sometimes SharePoint Timer Service (SPTimerV4) runs workflows. If you notice that the application showing the error is ULS logs in OWSTIMER.EXE, you should also include the authorized types in [SharePoint Hive Folder]\bin\OWSTIMER.EXE.config. The Hive Folder will change by version of SharePoint. For SharePoint 2016, it is normally at c:\program files\common files\microsoft shared\web server extensions\16. For 2013, at c:\program files\common files\microsoft shared\web server extensions\15.

Additional Information

My colleague Joe Rodgers, who is Sr. PFE, put together this PowerShell script: https://gist.github.com/joerodgers/2302b394796c865818839d843bae2dad

There are two scripts. Normally, the only necessary script is:

Add-CodeDomAuthorizedType.ps1

Uncomment this line to make the changes:

Add-CodeDomAuthorizedType

If you have Nintex workflows you should run like this:

Add-CodeDomAuthorizedType -IncludeNintexWorkflow

To undo the changes, run:

Remove-CodeDomAuthorizedType

The script needs to run only once on any WFE. All web.config files related to SharePoint on all servers will be modified. New web applications created after that will also include the changes. Even if a new WFE is added to the farm, the entries will also be included in web.config. The change is a permanent requirement from now on since the WF patch. You do not need to undo the change before applying the SharePoint patch addressing it.

There is a second script to update OWSTIMER.exe.config. This one should only run if you see the symptoms in ULS logs with process OWSTIMER.EXE. Otherwise, you do not need to update. if you have the problem though, you need to rerun the script if a new machine is added to the farm. No line needs to be uncommented for this one. The script name is:

Add-CodeDomAuthorizedTypeToOWSTimerConfig.ps1

Note

Microsoft is aware of this issue and patches for SharePoint 2010, 2013 and 2016 are being worked as of 9/17/2018. I will update when we have an ETA. I had confirmation from the product team on 9/18/2018 that this information and solution on this post is in the line with the future patch and it is the recommended action plan until the patch is out. If anything change, I will update the post.

Note 2

Some people using third-party workflows (like Nintex) need to also include this:

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeTypeReferenceExpression” Authorized=”True” />

Using the script, you need to add to the line defining types (line 24):

CodeTypeReferenceExpression

Example:
$typeNames = @( CodeBinaryOperatorExpression, CodePrimitiveExpression, CodeMethodInvokeExpression, CodeMethodReferenceExpression, CodeFieldReferenceExpression,CodeThisReferenceExpression, CodePropertyReferenceExpression“, “CodeTypeReferenceExpression”)

Note 3

Joe updated his script to add a switch for Nintex workflows.

Call this way to include the extra type required by Nintex:

Add-CodeDomAuthorizedType -IncludeNintexWorkflow

(all credit to Rodney Viana for this information)

Save as site template missing or greyed out

(This post was written using a SharePoint 2010 environment)

I recently found myself needing to create a site template for a customer who required multiple sites created following a base format. So I decided to create a site template to achieve this to speed up the site creation process for said customer.

At first I went into the site settings of the site that the template would be based off to create it. I looked under look and feel in site settings, but alas the option to save as site template wasn’t there!

site-actions-no-site-template

Next, I accessed the site via SharePoint Designer, looked under the site ribbon menu and would you believe the save as template button was greyed out there also!

SPD-site-template-button-greyed

After researching online, it appears there can be problems creating templates from publishing sites in SharePoint 2010, however I found that following the steps below enabled me to create a new site template and activate it not problem!

  • Navigate to the site collection/ sub-site you wish to create the template from
  • Add the following to the URL at the end of the site name

/_layouts/savetmpl.aspx

  • Go back to the site collection or sub-site you wish to create new sites under
  • Site Actions – Site Settings – Look and Feel – Page Layouts and site templates
site-settings-site-templates
  • Find your newly created site template within the subsite templates list
  • Press Add
  • Press OK
adding-subsite-templates

Please note: if your sub-site is inheriting from the parent site, you will need to navigate to the parent and add the subsite template there for it to be available.

UPDATE 27/08/2019

Saving as site template is not a supported feature of SharePoint Online for modern sites. You will find for modern sites that this option is not available, but you have the ability to re-instate it via activating a feature. The official stance from Microsoft is:

Just to be clear. Using save-site-as-a-template with modern team sites or publishing sites is NOT supported. You can always find aways with tenant admin permissions to achieve things, but the fact that it’s doable does not mean it’s supported.

Any sites created with the template which has been produced with this method can have unexpected issues in the future, so provided steps should never be used.

Best regards,

Vesa Juvonen, Senior Program Manager, SharePoint Engineering, Microsoft.

UPDATE 27/08/2019

Saving as site template is not a supported feature of SharePoint Online for modern sites. You will find for modern sites that this option is not available, but you have the ability to re-instate it via activating a feature. The official stance from Microsoft is:

” Just to be clear. Using save-site-as-a-template with modern team sites or publishing sites is NOT supported. You can always find aways with tenant admin permissions to achieve things, but the fact that it’s doable does not mean it’s supported.

Any sites created with the template which has been produced with this method can have unexpected issues in the future, so provided steps should never be used. “


– Vesa Juvonen, Senior Program Manager, SharePoint Engineering, Microsoft.

Hide a SharePoint list or library from view all site contents

Have you ever been asked to hide a list or library from a SharePoint site? If so, you go straight for selecting ‘no’ to displaying the list or library on the Quick Launch or removing it from the navigation. However, your eagle eyed users notice the handy view all site contents option and see that it is still listed there – they want it gone!

Luckily, all you need is SharePoint Designer and it is as simple as a click of a button…

(These steps were created using SharePoint Server 2010)

  • Open the site that where list or library resides in SharePoint Designer
  • Under Lists and Libraries – Select the list or library you wish to hide
  • On the main list settings page – find the Settings section
    SPD
  • Check the Hide from browser option
    hidefrombrowser

Thats it! when you option the view all site content page now, that list or library will no longer be showing. Also, if you want to re-instate it at a later date, just un-check the box and it will re-appear.

This also works for SharePoint 2013, 2016 and SharePoint Online, under the site contents page.