fundamentals

https://img.shields.io/pypi/pyversions/fundamentals https://img.shields.io/pypi/v/fundamentals https://img.shields.io/github/license/thespacedoctor/fundamentals https://img.shields.io/pypi/dm/fundamentals

http://167.99.90.204:8080/buildStatus/icon?job=fundamentals%2Fmaster&subject=build%20master http://167.99.90.204:8080/buildStatus/icon?job=fundamentals%2Fdevelop&subject=build%20dev https://cdn.jsdelivr.net/gh/thespacedoctor/fundamentals@master/coverage.svg https://readthedocs.org/projects/fundamentals/badge/?version=master https://img.shields.io/github/issues/thespacedoctor/fundamentals/type:%20bug?label=bug%20issues

Some fundamental tools required by most self-respecting python-packages bundled in one place. Very opinionated project setup tools including logging, plain-text settings files and database connections.

Documentation for fundamentals is hosted by Read the Docs (development version and master version). The code lives on github. Please report any issues you find here.

API Reference

Modules

fundamentals.commonutils

Common tools used thoughout the fundamentals package

fundamentals.download

Tools used to perform download of files and HTML pages from the web

fundamentals.files

Tools for working with files and folders

fundamentals.mysql

Some handy mysql database query and insertion tools

fundamentals.nose2_plugins

plugins for nose2 unit testing

fundamentals.renderer

*Render python objects as various list and markup formats *

fundamentals.logs

Logger setup for python projects

fundamentals.times

Some time functions to be used with logging etc

Classes

fundamentals.files.fileChunker

The fileChunker iterator - iterate over large line-based files to reduce memory footprint

fundamentals.logs.GroupWriteRotatingFileHandler

rotating file handler for logging

fundamentals.logs.emptyLogger

A fake logger object so user can set ``log=False`` if required

fundamentals.mysql.database

a database object that can setup up a ssh tunnel (optional) and a database connection

fundamentals.mysql.sqlite2mysql

Take a sqlite database file and copy the tables within it to a MySQL database

fundamentals.mysql.yaml_to_database

Take key-values from yaml files including a tablename(s) and add them to a mysql database table

fundamentals.renderer.list_of_dictionaries

*The dataset object is a list of python dictionaries.

fundamentals.tools

common setup methods & attributes of the main function in cl-util

fundamentals.utKit

Default setup for fundamentals style unit-testing workflow (all tests base on nose module)

Functions

fundamentals.commonutils.getpackagepath

Get the root path for this python package

fundamentals.download.append_now_datestamp_to_filename

append the current datestamp to the end of the filename (before the extension).

fundamentals.download.extract_filename_from_url

get the filename from a URL.

fundamentals.download.get_now_datetime_filestamp

A datetime stamp to be appended to the end of filenames: ‘YYYYMMDDtHHMMSS’

fundamentals.download.multiobject_download

get multiple url documents and place them in specified download directory/directories

fundamentals.files.list_of_dictionaries_to_mysql_inserts

Convert a python list of dictionaries to pretty csv output

fundamentals.files.recursive_directory_listing

list directory contents recursively.

fundamentals.files.tag

Add tags and ratings to your macOS files and folders

fundamentals.fmultiprocess

multiprocess pool

fundamentals.logs.console_logger

Setup and return a console logger

fundamentals.logs.setup_dryx_logging

setup dryx style python logging

fundamentals.mysql.convert_dictionary_to_mysql_table

convert dictionary to mysql table

fundamentals.mysql.directory_script_runner

A function to run all mysql scripts in a given directory (in a modified date order, oldest first) and then act on the script files in accordance with the succcess or failure of their execution

fundamentals.mysql.get_database_table_column_names

get database table column names

fundamentals.mysql.insert_list_of_dictionaries_into_database_tables

insert list of dictionaries into database tables

fundamentals.mysql.readquery

Given a mysql query, read the data from the database and return the results as a list of dictionaries (database rows)

fundamentals.mysql.setup_database_connection

Start a database connection using settings in yaml file

fundamentals.mysql.table_exists

Probe a database to determine if a given table exists

fundamentals.mysql.writequery

Execute a MySQL write command given a sql query

fundamentals.times.calculate_time_difference

Return the time difference between two dates as a string

fundamentals.times.datetime_relative_to_now

*convert date to a relative datetime (e.g.

fundamentals.times.get_now_sql_datetime

A datetime stamp in MySQL format: ‘YYYY-MM-DDTHH:MM:SS’

Release Notes

v2.2.4 - May 28, 2020

  • enhancement: allowing ‘~’ as home directory for filepaths in all settings file parameters - will be converted when initially read

v2.2.3 - May 26, 2020

  • fixed: delimiters catered for in sql scripts

v2.2.2 - May 25, 2020

  • refactor:list_of_dictionaries now returns bytes decoded into UTF-8 string when rendered to other mark-up flavour.

  • refactor: moved module level numpy imports so that packages with fundamentals as a dependency do not have numpy as a needless dependency

v2.2.1 - May 13, 2020

  • fixed:directory_script_runner function databaseName parameter changed to be optional

v2.2.0 - May 13, 2020

  • feature: new execute_mysql_script function that allows execution of a sql script directly from file

  • refactor: added a dbConn to the directory script runner

v2.1.7 - May 4, 2020

  • fixed: inspect module depreciation warnings stopped

  • fixed: MySQL error messages printed to stdout (previously hidden)

v2.1.3 - April 17, 2020

  • enhancement: cleaned up docs theme and structure. More documentation to come.