Skip to content

Changelog

Release notes

0.14.3 -- Sep 23, 2024

  • Added - dj.Top restriction - PR #1024) PR #1084
  • Fixed - Added encapsulating double quotes to comply with DOT language - PR #1177
  • Added - Datajoint python CLI (#940) - PR #1095
  • Added - Ability to set hidden attributes on a table - PR #1091
  • Added - Ability to specify a list of keys to populate - PR #989
  • Fixed - fixed topological sort #1057 - PR #1184
  • Fixed - .parts() not always returning parts #1103 - PR #1184
  • Changed - replace setup.py with pyproject.toml - PR #1183
  • Changed - disable add_hidden_timestamp configuration option by default - PR #1188

0.14.2 -- Aug 19, 2024

  • Added - Migrate nosetests to pytest - PR #1142
  • Added - Codespell GitHub Actions workflow
  • Added - GitHub Actions workflow to manually release docs
  • Changed - Update datajoint/nginx to v0.2.6
  • Changed - Migrate docs from https://docs.datajoint.org/python to https://datajoint.com/docs/core/datajoint-python
  • Fixed - DevContainer configuration - PR #1115
  • Fixed - Updated set_password to work on MySQL 8 - PR #1106
  • Added - Missing tests for set_password - PR #1106
  • Changed - Returning success count after the .populate() call - PR #1050
  • Fixed - Autopopulate.populate excludes reserved jobs in addition to ignore and error jobs
  • Fixed - Issue #1159 (cascading delete) - PR #1160
  • Changed - Minimum Python version for Datajoint-Python is now 3.8 PR #1163
  • Fixed - docker compose commands in CI #1164
  • Changed - Default delete behavior now includes masters of part tables - PR #1158

0.14.1 -- Jun 02, 2023

  • Fixed - Fix altering a part table that uses the "master" keyword - PR #991
  • Fixed - .ipynb output in tutorials is not visible in dark mode (#1078) PR #1080
  • Fixed - preview table font for darkmode PR #1089
  • Changed - Readme to update links and include example pipeline image
  • Changed - Docs to add landing page and update navigation
  • Changed - .data method to .stream in the get() method for S3 (external) objects PR #1085
  • Fixed - Docs to rename create_virtual_module to VirtualModule
  • Added - Skeleton from datajoint-company/datajoint-docs repository for docs migration
  • Added - Initial pytest for test_connection

0.14.0 -- Feb 13, 2023

  • Added - json data type (#245) PR #1051
  • Fixed - Activating a schema requires all tables to exist even if create_tables=False PR #1058
  • Changed - Populate call with reserve_jobs=True to exclude error and ignore keys - PR #1062
  • Added - Support for inserting data with CSV files - PR #1067
  • Changed - Switch testing image from pydev to djtest PR #1012
  • Added - DevContainer development environment compatible with GH Codespaces PR 1071
  • Fixed - Convert lingering prints by replacing with logs PR #1073
  • Changed - table.progress() defaults to no stdout PR #1073
  • Changed - table.describe() defaults to no stdout PR #1073
  • Deprecated - table._update() PR #1073
  • Deprecated - old-style foreign key syntax PR #1073
  • Deprecated - dj.migrate_dj011_external_blob_storage_to_dj012() PR #1073
  • Added - Method to set job keys to "ignore" status - PR #1068

0.13.8 -- Sep 21, 2022

  • Added - New documentation structure based on markdown PR #1052
  • Fixed - Fix queries with backslashes (#999) PR #1052

0.13.7 -- Jul 13, 2022

  • Fixed - Fix networkx incompatible change by version pinning to 2.6.3 (#1035) PR #1036
  • Added - Support for serializing numpy datetime64 types (#1022) PR #1036
  • Changed - Add traceback to default logging PR #1036

0.13.6 -- Jun 13, 2022

  • Added - Config option to set threshold for when to stop using checksums for filepath stores. PR #1025
  • Added - Unified package level logger for package (#667) PR #1031
  • Changed - Swap various datajoint messages, warnings, etc. to use the new logger. (#667) PR #1031
  • Fixed - Fix query caching deleting non-datajoint files PR #1027
  • Changed - Minimum Python version for Datajoint-Python is now 3.7 PR #1027

0.13.5 -- May 19, 2022

  • Changed - Import ABC from collections.abc for Python 3.10 compatibility
  • Fixed - Fix multiprocessing value error (#1013) PR #1026

0.13.4 -- Mar, 28 2022

  • Added - Allow reading blobs produced by legacy 32-bit compiled mYm library for matlab. PR #995
  • Fixed - Add missing jobs argument for multiprocessing PR #997
  • Added - Test for multiprocessing PR #1008
  • Fixed - Fix external store key name doesn't allow '-' (#1005) PR #1006
  • Added - Adopted black formatting into code base PR #998

0.13.3 -- Feb 9, 2022

  • Fixed - Fix error in listing ancestors, descendants with part tables.
  • Fixed - Fix Python 3.10 compatibility (#983) PR #972
  • Fixed - Allow renaming non-conforming attributes in proj (#982) PR #972
  • Added - Expose proxy feature for S3 external stores (#961) PR #962
  • Added - implement multiprocessing in populate (#695) PR #704, #969
  • Fixed - Dependencies not properly loaded on populate. (#902) PR #919
  • Fixed - Replace use of numpy aliases of built-in types with built-in type. (#938) PR #939
  • Fixed - Deletes and drops must include the master of each part. (#151, #374) PR #957
  • Fixed - ExternalTable.delete should not remove row on error (#953) PR #956
  • Fixed - Fix error handling of remove_object function in s3.py (#952) PR #955
  • Fixed - Fix regression issue with DISTINCT clause and GROUP_BY (#914) PR #963
  • Fixed - Fix sql code generation to comply with sql mode ONLY_FULL_GROUP_BY (#916) PR #965
  • Fixed - Fix count for left-joined QueryExpressions (#951) PR #966
  • Fixed - Fix assertion error when performing a union into a join (#930) PR #967
  • Changed ~jobs.error_stack from blob to mediumblob to allow error stacks >64kB in jobs (#984) PR #986
  • Fixed - Fix error when performing a union on multiple tables (#926) PR #964
  • Added - Allow optional keyword arguments for make() in populate() PR #971

0.13.2 -- May 7, 2021

  • Changed setuptools_certificate dependency to new name otumat
  • Fixed - Explicit calls to dj.Connection throw error due to missing host_input (#895) PR #907
  • Fixed - Correct count of deleted items. (#897) PR #912

0.13.1 -- Apr 16, 2021

  • Added None as an alias for IS NULL comparison in dict restrictions (#824) PR #893
  • Changed - Drop support for MySQL 5.6 since it has reached EOL PR #893
  • Fixed - schema.list_tables() is not topologically sorted (#838) PR #893
  • Fixed - Diagram part tables do not show proper class name (#882) PR #893
  • Fixed - Error in complex restrictions (#892) PR #893
  • Fixed - WHERE and GROUP BY classes are dropped on joins with aggregation (#898, #899) PR #893

0.13.0 -- Mar 24, 2021

  • Re-implement query transpilation into SQL, fixing issues (#386, #449, #450, #484, #558). PR #754
  • Re-implement cascading deletes for better performance. PR #839
  • Add support for deferred schema activation to allow for greater modularity. (#834) PR #839
  • Add query caching mechanism for offline development (#550) PR #839
  • Add table method .update1 to update a row in the table with new values (#867) PR #763, #889
  • Python datatypes are now enabled by default in blobs (#761). PR #859
  • Added permissive join and restriction operators @ and ^ (#785) PR #754
  • Support DataJoint datatype and connection plugins (#715, #729) PR 730, #735
  • Add dj.key_hash alias to dj.hash.key_hash (#804) PR #862
  • Default enable_python_native_blobs to True
  • Bugfix - Regression error on joins with same attribute name (#857) PR #878
  • Bugfix - Error when fetch1('KEY') when dj.config['fetch_format']='frame' set (#876) PR #880, #878
  • Bugfix - Error when cascading deletes in tables with many, complex keys (#883, #886) PR #839
  • Add deprecation warning for _update. PR #889
  • Add purge_query_cache utility. PR #889
  • Add tests for query caching and permissive join and restriction. PR #889
  • Drop support for Python 3.5 (#829) PR #861

0.12.9 -- Mar 12, 2021

  • Fix bug with fetch1 with dj.config['fetch_format']="frame". (#876) PR #880

0.12.8 -- Jan 12, 2021

  • table.children, .parents, .descendents, and ancestors can return queryable objects. PR #833
  • Load dependencies before querying dependencies. (#179) PR #833
  • Fix display of part tables in schema.save. (#821) PR #833
  • Add schema.list_tables. (#838) PR #844
  • Fix minio new version regression. PR #847
  • Add more S3 logging for debugging. (#831) PR #832
  • Convert testing framework from TravisCI to GitHub Actions (#841) PR #840

0.12.7 -- Oct 27, 2020

  • Fix case sensitivity issues to adapt to MySQL 8+. PR #819
  • Fix pymysql regression bug (#814) PR #816
  • Adapted attribute types now have dtype=object in all recarray results. PR #811

0.12.6 -- May 15, 2020

  • Add order_by to dj.kill (#668, #779) PR #775, #783
  • Add explicit S3 bucket and file storage location existence checks (#748) PR #781
  • Modify _update to allow nullable updates for strings/date (#664) PR #760
  • Avoid logging events on auxiliary tables (#737) PR #753
  • Add kill_quick and expand display to include host (#740) PR #741
  • Bugfix - pandas insert fails due to additional index field (#666) PR #776
  • Bugfix - delete_external_files=True does not remove from S3 (#686) PR #781
  • Bugfix - pandas fetch throws error when fetch_format='frame' PR #774

0.12.5 -- Feb 24, 2020

  • Rename module dj.schema into dj.schemas. dj.schema remains an alias for class dj.Schema. (#731) PR #732
  • dj.create_virtual_module is now called dj.VirtualModule (#731) PR #732
  • Bugfix - SSL KeyError on failed connection (#716) PR #725
  • Bugfix - Unable to run unit tests using nosetests (#723) PR #724
  • Bugfix - suppress_errors does not suppress loss of connection error (#720) PR #721

0.12.4 -- Jan 14, 2020

  • Support for simple scalar datatypes in blobs (#690) PR #709
  • Add support for the serial data type in declarations: alias for bigint unsigned auto_increment PR #713
  • Improve the log table to avoid primary key collisions PR #713
  • Improve documentation in README PR #713

0.12.3 -- Nov 22, 2019

  • Bugfix - networkx 2.4 causes error in diagrams (#675) PR #705
  • Bugfix - include table definition in doc string and help (#698, #699) PR #706
  • Bugfix - job reservation fails when native python datatype support is disabled (#701) PR #702

0.12.2 -- Nov 11, 2019

  • Bugfix - Convoluted error thrown if there is a reference to a non-existent table attribute (#691) PR #696
  • Bugfix - Insert into external does not trim leading slash if defined in dj.config['stores']['<store>']['location'] (#692) PR #693

0.12.1 -- Nov 2, 2019

  • Bugfix - AttributeAdapter converts into a string (#684) PR #688

0.12.0 -- Oct 31, 2019

  • Dropped support for Python 3.4
  • Support secure connections with TLS (aka SSL) PR #620
  • Convert numpy array from python object to appropriate data type if all elements are of the same type (#587) PR #608
  • Remove expression requirement to have additional attributes (#604) PR #604
  • Support for filepath datatype (#481) PR #603, #659
  • Support file attachment datatype (#480, #592, #637) PR #659
  • Fetch return a dict array when specifying as_dict=True for specified attributes. (#595) PR #593
  • Support of ellipsis in proj: query_expression.proj(.., '-movie') (#499) PR #578
  • Expand support of blob serialization (#572, #520, #427, #392, #244, #594) PR #577
  • Support for alter (#110) PR #573
  • Support for conda install datajoint via conda-forge channel (#293)
  • dj.conn() accepts a port keyword argument (#563) PR #571
  • Support for UUID datatype (#562) PR #567
  • query_expr.fetch("KEY", as_dict=False) returns results as np.recarray(#414) PR #574
  • dj.ERD is now called dj.Diagram (#255, #546) PR #565
  • dj.Diagram underlines "distinguished" classes (#378) PR #557
  • Accept alias for supported MySQL datatypes (#544) PR #545
  • Support for pandas in fetch (#459, #537) PR #534
  • Support for ordering by "KEY" in fetch (#541) PR #534
  • Add config to enable python native blobs PR #672, #676
  • Add secure option for external storage (#663) PR #674, #676
  • Add blob migration utility from DJ011 to DJ012 PR #673
  • Improved external storage - a migration script needed from version 0.11 (#467, #475, #480, #497) PR #532
  • Increase default display rows (#523) PR #526
  • Bugfixes (#521, #205, #279, #477, #570, #581, #597, #596, #618, #633, #643, #644, #647, #648, #650, #656)
  • Minor improvements (#538)

0.11.3 -- Jul 26, 2019

  • Fix incompatibility with pyparsing 2.4.1 (#629) PR #631

0.11.2 -- Jul 25, 2019

  • Fix #628 - incompatibility with pyparsing 2.4.1

0.11.1 -- Nov 15, 2018

  • Fix ordering of attributes in proj (#483, #516)
  • Prohibit direct insert into auto-populated tables (#511)

0.11.0 -- Oct 25, 2018

  • Full support of dependencies with renamed attributes using projection syntax (#300, #345, #436, #506, #507)
  • Rename internal class and module names to comply with terminology in documentation (#494, #500)
  • Full support of secondary indexes (#498, 500)
  • ERD no longer shows numbers in nodes corresponding to derived dependencies (#478, #500)
  • Full support of unique and nullable dependencies (#254, #301, #493, #495, #500)
  • Improve memory management in populate (#461, #486)
  • Fix query errors and redundancies (#456, #463, #482)

0.10.1 -- Aug 28, 2018

  • Fix ERD Tooltip message (#431)
  • Networkx 2.0 support (#443)
  • Fix insert from query with skip_duplicates=True (#451)
  • Sped up queries (#458)
  • Bugfix in restriction of the form (A & B) * B (#463)
  • Improved error messages (#466)

0.10.0 -- Jan 10, 2018

  • Deletes are more efficient (#424)
  • ERD shows table definition on tooltip hover in Jupyter (#422)
  • S3 external storage
  • Garbage collection for external sorage
  • Most operators and methods of tables can be invoked as class methods rather than instance methods (#407)
  • The schema decorator object no longer requires locals() to specify the context
  • Compatibility with pymysql 0.8.0+
  • More efficient loading of dependencies (#403)

0.9.0 -- Nov 17, 2017

  • Made graphviz installation optional
  • Implement file-based external storage
  • Implement union operator +
  • Implement file-based external storage

0.8.0 -- Jul 26, 2017

Documentation and tutorials available at https://docs.datajoint.io and https://tutorials.datajoint.io

  • improved the ERD graphics and features using the graphviz libraries (#207, #333)
  • improved password handling logic (#322, #321)
  • the use of the contents property to populate tables now only works in dj.Lookup classes (#310).
  • allow suppressing the display of size of query results through the show_tuple_count configuration option (#309)
  • implemented renamed foreign keys to spec (#333)
  • added the limit keyword argument to populate (#329)
  • reduced the number of displayed messages (#308)
  • added size_on_disk property for dj.Schema() objects (#323)
  • job keys are entered in the jobs table (#316, #243)
  • simplified the fetch and fetch1 syntax, deprecating the fetch[...] syntax (#319)
  • the jobs tables now store the connection ids to allow identifying abandoned jobs (#288, #317)

0.5.0 (#298) -- Mar 8, 2017

  • All fetched integers are now 64-bit long and all fetched floats are double precision.
  • Added dj.create_virtual_module

0.4.10 (#286) -- Feb 6, 2017

  • Removed Vagrant and Readthedocs support
  • Explicit saving of configuration (issue #284)

0.4.9 (#285) -- Feb 2, 2017

  • Fixed setup.py for pip install

0.4.7 (#281) -- Jan 24, 2017

  • Fixed issues related to order of attributes in projection.

0.4.6 (#277) -- Dec 22, 2016

  • Proper handling of interruptions during populate

0.4.5 (#274) -- Dec 20, 2016

  • Populate reports how many keys remain to be populated at the start.

0.4.3 (#271) -- Dec 6, 2016

  • Fixed aggregation issues (#270)
  • datajoint no longer attempts to connect to server at import time
  • dropped support of view (reversed #257)
  • more elegant handling of insufficient privileges (#268)

0.4.2 (#267) -- Dec 6, 2016

  • improved table appearance in Jupyter

0.4.1 (#266) -- Oct 28, 2016

  • bugfix for very long error messages

0.3.9 -- Sep 27, 2016

  • Added support for datatype YEAR
  • Fixed issues with dj.U and the aggr operator (#246, #247)

0.3.8 -- Aug 2, 2016

  • added the _update method in base_relation. It allows updating values in existing tuples.
  • bugfix in reading values of type double. Previously it was cast as float32.

0.3.7 -- Jul 31, 2016

  • added parameter ignore_extra_fields in insert
  • insert(..., skip_duplicates=True) now relies on SELECT IGNORE. Previously it explicitly checked if tuple already exists.
  • table previews now include blob attributes displaying the string

0.3.6 -- Jul 30, 2016

  • bugfix in schema.spawn_missing_classes. Previously, spawned part classes would not show in ERDs.
  • dj.key now causes fetch to return as a list of dicts. Previously it was a recarray.

0.3.5

  • dj.set_password() now asks for user confirmation before changing the password.
  • fixed issue #228

0.3.4

  • Added method the ERD.add_parts method, which adds the part tables of all tables currently in the ERD.
  • ERD() + arg and ERD() - arg can now accept table classes as arg.

0.3.3

  • Suppressed warnings (redirected them to logging). Previoiusly, scipy would throw warnings in ERD, for example.
  • Added ERD.from_sequence as a shortcut to combining the ERDs of multiple sources
  • ERD() no longer text the context argument.
  • ERD.draw() now takes an optional context argument. By default uses the caller's locals.

0.3.2

  • Fixed issue #223: insert can insert relations without fetching.
  • ERD() now takes the context argument, which specifies in which context to look for classes. The default is taken from the argument (schema or table).
  • ERD.draw() no longer has the prefix argument: class names are shown as found in the context.