File

Collection of DIRAC useful file related modules.

Warning

By default on Error they return None.

DIRAC.Core.Utilities.File.checkGuid(guid)

Checks whether a supplied GUID is of the correct format. The guid is a string of 36 characters [0-9A-F] long split into 5 parts of length 8-4-4-4-12.

Warning

As we are using GUID produced by various services and some of them could not follow convention, this function is passing by a guid which can be made of lower case chars or even just have 5 parts of proper length with whatever chars.

Parameters:

guid (string) – string to be checked

Returns:

True (False) if supplied string is (not) a valid GUID.

DIRAC.Core.Utilities.File.cleanDirectory(workDir: str | PathLike[str], maxSecs: int | float | None = None, filePatterns: list[str] = [], maxDepth: int = 0, callbackFn: Callable[[Path], bool] | None = None, delEmptyDirs: bool = False) → list[str]

Recursively clean files older than a threshold.

Walks workDir bottom-up and deletes (or invokes callbackFn on) regular files that are older than maxSecs seconds which match the filePatterns glob. Empty directories can also be removed with delEmptyDirs``=``True.

Parameters:

  • workDir – directory to scan

  • maxSecs – age threshold in seconds (files older are deleted); pass None to skip the age filter entirely

  • filePatterns – list of globs, only files matching this will be considered

  • maxDepth – maximum directory depth to process (0 = unlimited, 1 = root only)

  • callbackFn – If None files will be unlinked, otherwise this function will be called for matchin files instead. Function should take a single``Path`` object argument. Returning True indicates the file was processed without error. Returning False will add the path onto the list of failed files.

  • delEmptyDirs – if True, delete directories that are empty after file cleanup. All directories are considered, the filePatterns glob is not used to filter these.

Returns:

list of file (and dir) paths that could not be deleted (empty on success)

DIRAC.Core.Utilities.File.convertSizeUnits(size, srcUnit, dstUnit)

Converts a number from a given source unit to a destination unit.

Example

In [1]: convertSizeUnits(1024, ‘B’, ‘kB’) Out[1]: 1

In [2]: convertSizeUnits(1024, ‘MB’, ‘kB’) Out[2]: 1048576

Parameters:

  • size – number to convert

  • srcUnit – unit of the number. Any of ( ‘B’, ‘kB’, ‘MB’, ‘GB’, ‘TB’, ‘PB’)

  • dstUnit – unit expected for the return. Any of ( ‘B’, ‘kB’, ‘MB’, ‘GB’, ‘TB’, ‘PB’)

Returns:

the size number converted in the dstUnit. In case of problem -sys.maxint is returned (negative)

DIRAC.Core.Utilities.File.generateGuid(checksum, checksumtype)

Generate a GUID based on the file checksum

DIRAC.Core.Utilities.File.getGlobbedFiles(files)

Get list of files or a single file. Globs the parameter to allow regular expressions.

Params list files:

list or tuple of strings of files

DIRAC.Core.Utilities.File.getGlobbedTotalSize(files)

Get total size of a list of files or a single file. Globs the parameter to allow regular expressions.

Params list files:

list or tuple of strings of files

DIRAC.Core.Utilities.File.getMD5ForFiles(fileList)

Calculate md5 for the content of all the files.

Parameters:

fileList (list) – list of paths

DIRAC.Core.Utilities.File.getSize(fileName: PathLike) → int

Get size of a file.

Parameters:

fileName (string) – name of file to be checked

The os module claims only OSError can be thrown, but just for curiosity it’s catching all possible exceptions.

Warning

On any exception it returns -1.

DIRAC.Core.Utilities.File.makeGuid(fileName=None)

Utility to create GUID. If a filename is provided the GUID will correspond to its content’s hexadecimal md5 checksum. Otherwise a random seed is used to create the GUID. The format is capitalized 8-4-4-4-12.

Warning

Could return None in case of OSError or IOError.

Parameters:

fileName (string) – name of file

DIRAC.Core.Utilities.File.mkDir(path, mode=None)

Emulate ‘mkdir -p path’ (if path exists already, don’t raise an exception)

Parameters:

  • path (str) – directory hierarchy to create

  • mode (int) – Use this mode as the mode for new directories, use python default if None.

Protected creation of symbolic link

DIRAC.Core.Utilities.File.secureOpenForWrite(filename=None, *, text=True)

Securely open a file for writing.

If filename is not provided, a file is created in tempfile.gettempdir(). The file always created with mode 600.

Parameters:

filename (string) – name of file to be opened