autour_de_minuit/blender-asset-tracer
8
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 4 Wiki Activity
253 Commits 1 Branch 41 Tags
e7cd6ab70dad09b95ea0bcaefbb6003f97eeb9ac
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Sybren A. Stüvel e7cd6ab70d Fix external smoke caches not being packed 2019-09-27 12:19:22 +02:00
Sybren A. Stüvel 5988f3599a Fix T69976: BAT does not work with FluidSimModifier in Blender 2.81 
The `point_cache` pointer was removed from Blender in
{rBd663ec48c06a083de74e90dd9c605e98b28baa37}, and was unused before that
commit.
 2019-09-26 12:26:42 +02:00
Sybren A. Stüvel af63f9c3c8 Allow setting Shaman JWT token in SHAMAN_JWT_TOKEN environment variable 
I also removed the unused `shaman/auth.py` file.
 2019-09-24 11:49:48 +02:00
Sybren A. Stüvel 910945e44d Added note about RNA and DNA in README.md 2019-08-12 11:47:06 +02:00
Sybren A. Stüvel 40c27641e5 Extended the code example in README.md a bit 2019-08-12 11:38:41 +02:00
Sybren A. Stüvel a07c25597f Added a code example to the README.md 2019-08-12 10:55:41 +02:00
Sybren A. Stüvel 113b0c9bb8 Fix T65904: External files with same path on different drives are packed once 
External files with the same path on different drives are packed as a
single file. In this commit the drive letter is taken into account when
determining the path inside `_outside_project`, so that they are distinct.
 2019-06-26 14:37:29 +02:00
Sybren A. Stüvel f0836417a4 Bumped version to 1.2-dev0 2019-06-12 11:49:27 +02:00
Sybren A. Stüvel 3ae8b512df Updated changelog 2019-06-12 11:49:18 +02:00
Sybren A. Stüvel b48978f067 Moved from Pipenv to Poetry 
All the Blender Institute Python webprojects are moving to Poetry; it works
better than Pipenv in various regards.
 2019-06-07 12:49:59 +02:00
Sybren A. Stüvel f40f7585ed Prevent error when modifying actions dict 2019-06-07 12:35:41 +02:00
Sybren A. Stüvel 37fc7f467a Fix exception when having project on a mapped network drive in Windows 2019-06-07 12:35:41 +02:00
Sybren A. Stüvel a864ccdf70 Updated changelog 2019-06-05 14:14:00 +02:00
Sybren A. Stüvel c2e3aea3f6 Bumped version to 1.1.1 2019-04-18 12:04:51 +02:00
Sybren A. Stüvel acd5a18bed Import urllib3 from requests.packages.urllib3 
This is for compatibility with Blender 2.79.
 2019-04-18 12:04:03 +02:00
Sybren A. Stüvel 24eaacff84 Declare requests as dependency 
It's used by the Shaman interface only, so other BAT functions could be
used without it as well.
 2019-04-18 11:47:28 +02:00
Sybren A. Stüvel 5d376a8054 Allow testing with Tox 2019-04-18 11:43:18 +02:00
Sybren A. Stüvel 9fdfbc9bb3 Bumped version to 1.1 2019-03-25 17:46:53 +01:00
Sybren A. Stüvel be9798cb1c Marked version 1.1 as released 2019-03-25 17:46:38 +01:00
Sybren A. Stüvel 4f05b2d481 Updated URL to Shaman 
The Shaman server as standalone component is no more, it's been integrated
into Flamenco Manager.
 2019-03-25 17:46:28 +01:00
Sybren A. Stüvel b418aaf3c4 Bumped version to 1.1.dev2 2019-03-20 13:58:23 +01:00
Sybren A. Stüvel 4964745dee Support Alembic files from linked library 
We already supported Alembic files in the top-level blend file, but now we
also support finding Alembic files in linked-in libraries.
 2019-03-20 13:57:44 +01:00
Sybren A. Stüvel c718fb3a41 Support the Mesh Sequence Cache modifier (a.k.a. Alembic files) 2019-03-20 11:14:23 +01:00
Sybren A. Stüvel 6ac117cd37 Formatting 2019-03-20 11:12:47 +01:00
Sybren A. Stüvel e9fcdb0191 Use exponential backoff when retrying HTTP connections 2019-03-06 16:21:30 +01:00
Sybren A. Stüvel 44ee553c0a Shaman: added some timeouts to HTTP requests 2019-03-06 16:00:46 +01:00
Sybren A. Stüvel c2a8ff18f8 Bumped version to 1.1-dev1 2019-03-06 13:40:32 +01:00
Sybren A. Stüvel 10abeaea49 Fix ValueError when not requesting a checkout 2019-03-06 12:10:55 +01:00
Sybren A. Stüvel dabb249e9b Python 3.5 fixes 
`typing.Deque` was introduced in Python 3.6, so we can only use it when
we no longer support Blender 2.79.
 2019-03-01 15:12:24 +01:00
Sybren A. Stüvel 8c822c5b76 Updated changelog 2019-03-01 14:32:50 +01:00
Sybren A. Stüvel 7a7304d7a3 Shaman: Include original filename when uploading file 2019-03-01 14:07:24 +01:00
Sybren A. Stüvel a2f9a7b484 Shaman changed response does-not-exist → file-unknown 
Instead of 'does-not-exist' we'll now receive 'file-unknown' (see Shaman
commit 44efa6b39bddc703a14aa68316b092866813c38a)
 2019-03-01 14:07:24 +01:00
Sybren A. Stüvel b22b9da5d0 Shaman: Moved URL parsing into a separate function 
This allows other code to just call that function, rather than copying
the behaviour.
 2019-03-01 14:07:24 +01:00
Sybren A. Stüvel f34bf7c66f When aborting, make sure the reason is available 
This allows a GUI to display the reason of abortion.
 2019-03-01 14:07:24 +01:00
Sybren A. Stüvel 6104bd6078 Shaman: Make checkout location available to the Packer 2019-03-01 14:07:24 +01:00
Sybren A. Stüvel 14417b2d73 Record bytes & files uploaded on Transferer 2019-03-01 14:07:24 +01:00
Sybren A. Stüvel 9074421ec3 Shaman server support 
The Shaman server is a file storage system that identifies files by
SHA256sum and file length. BAT can send packs there by only uploading
changed/new files. The BAT pack is reproduced at the Shaman server's
checkout directory by creating symlinks to the files in its file
storage.

Retrying sending files:

When we can defer uploading a file (that is, when we have other files to
upload as well, and we could send the current file at a later moment) we
send an `X-Shaman-Can-Defer-Upload: true` header in the file upload
request. In that case, when someone else is already uploading that file,
a `208 Already Reported` response is sent and the connection is closed.
Python's Requests library unfortunately won't give us that response if
we're still streaming the request, and raise a ConnectionError exception
instead. This exception can mean two things:

- If the `X-Shaman-Can-Defer-Upload: true` header was sent: someone else
  is currently uploading that file, so defer it.
- If that header was not sent: that file is already completely uploaded
  and does not need to be uploaded again.

Instead of retrying each failed file, after a few failures we now just
resend the definition file to get a new list of files to upload, then
send those. This should considerably reduce the number of HTTP calls
when multiple clients are uploading the same set of files.
 2019-03-01 14:07:24 +01:00
Sybren A. Stüvel 3dd4ff5ba6 Bumped version to 1.1-dev0 2019-03-01 14:07:12 +01:00
Sybren A. Stüvel c8337f5098 Bumped version to 1.0 
This is the same version as 0.99, just released as 1.0 to note the
maturity of the code (it's been used successfully for months at the Blender
Animation Studio).
 2019-03-01 14:03:55 +01:00
Sybren A. Stüvel 8dede717c6 Bumped version to 0.99 2019-02-26 16:45:43 +01:00
Sybren A. Stüvel add5cf930b Include error message with error status 2019-02-26 16:35:55 +01:00
Sybren A. Stüvel 03fb4da583 Convert target path from Path to str & PurePath 
The target path is just read as string from the CLI now, to allow more
complex targets (such as URLs) that don't directly map to a path.

The Packer subclass now handles the conversion from that string to a
`pathlib.PurePath`, and specific subclasses & transfer classes can convert
those to a `pathlib.Path` to perform actual filesystem operations when
necessary.
 2019-02-26 16:35:08 +01:00
Sybren A. Stüvel 754edd8711 Update dependencies 2019-02-26 16:35:08 +01:00
Sybren A. Stüvel c4a0d5925a Use 'bat-' prefix when creating temporary directories 2019-02-26 16:35:08 +01:00
Sybren A. Stüvel e9e1cfd8f5 Removed unused imports 2019-02-26 14:49:54 +01:00
Sybren A. Stüvel d5d9c7f52f Clarified comment 2019-02-12 12:48:02 +01:00
Sybren A. Stüvel dcc992f1cd Bumped version to 0.9 2019-02-12 12:31:51 +01:00
Sybren A. Stüvel 66681a69df Fixed crash where collection children are ID blocks instead of GR blocks 
Not sure when/how that happens, but it happened on a lighting file of the
Spring project, when linking in a nested collection of which parts where
also linked in from other blend files.
 2019-02-12 12:31:05 +01:00
Sybren A. Stüvel bf7d2b1e27 Added some more unit tests for BlendPath 2019-02-04 15:42:40 +01:00
Sybren A. Stüvel ee29184b3c Removed unused FileCopierPool class 2019-02-04 15:34:29 +01:00
Sybren A. Stüvel 1434a9ee5d Added --sha256 and --timing options to bat list 
These are only temporary features for aiding in development.
 2019-02-04 15:30:52 +01:00
Sybren A. Stüvel 55a14eca1b Removed some extraneous commas 2019-02-04 15:17:14 +01:00
Sybren A. Stüvel daab1df826 Use a priority queue instead of repeatedly sorting a deque 2019-02-04 15:17:07 +01:00
Sybren A. Stüvel efbd7c1cb2 Bumped version to 0.8 2019-01-02 16:06:38 +01:00
Sybren A. Stüvel f7fb4b13ca Test packing a blend file with Windows relative paths 2019-01-02 15:41:56 +01:00
Sybren A. Stüvel b79a5dc491 Avoid repeating/constructing the same path 2019-01-02 15:39:09 +01:00
Sybren A. Stüvel 7df73d6c69 Configure test logging at import time (rather than at setup time) 2019-01-02 15:38:34 +01:00
Sybren A. Stüvel 20d7ea08cc Workaround for Windows failing on glob patterns in path.resolve() 
Windows fails with an OSError when `somepath.resolve()` is called and
`somepath` contains a glob pattern. As a workaround, we now `resolve()`
the parent directory, and put the filename at its end. This only works
when the glob pattern is in the filename, which is the case for BAT-
generated globs.
 2019-01-02 15:14:50 +01:00
Sybren A. Stüvel d222fea960 Explicitly write pack-info.txt as UTF-8 2019-01-02 15:14:50 +01:00
Sybren A. Stüvel 772a5a1b1c Replace → with -> 
Apparently Windows still isn't using UTF-8 by default, and fails when
handling these log entries.
 2019-01-02 15:14:50 +01:00
Sybren A. Stüvel 5a41167a09 Missing test teardowns 
This kept a file pointer open while trying to delete the file.
 2019-01-02 15:14:50 +01:00
Sybren A. Stüvel d81fe590db BlendPath: don't use os.path.join() for slash notations 
The BlendPath should just use forward slashes, and not be depending on the
current platform.
 2019-01-02 15:14:50 +01:00
Sybren A. Stüvel 0e392f27c9 BlendPath.__repr__ implemented 2019-01-02 15:14:50 +01:00
Sybren A. Stüvel 19216cb12e BlendPath.as_path() now refuses to convert blendfile-relative paths 
When a path starts with b'//' it will not be converted to a PurePath, as
the handling of such filenames is platform dependent (Windows handles those
weirdly, like appending a slash to any path).
 2019-01-02 15:14:50 +01:00
Sybren A. Stüvel 5635895d0c Attempt at making BAT work on Windows 
All BlendPath instances will use forward slashes, and there should be
more use of PurePosixPath instead of Path.
 2019-01-02 15:14:50 +01:00
Sybren A. Stüvel 37578811d7 Upgraded dependencies 2018-12-20 14:23:12 +01:00
Sybren A. Stüvel 4bc5484c23 Fixed MyPy issue 2018-12-20 14:23:06 +01:00
Sybren A. Stüvel c331b99dc6 Fixed leaking file descriptors in unit tests 
These weren't any issue in Linux, but Windows doesn't like it when you
try to delete a still-opened file.
 2018-12-20 14:04:23 +01:00
Sybren A. Stüvel fd316c4594 Bumped version to 0.7 2018-12-06 10:02:40 +01:00
Sybren A. Stüvel 37ad8f8dfc Added --relative-only option to bat pack command 
This makes BAT skip assets that are referred to with an absolute path.
It is assumed that the receiver of the BAT pack can access those assets
at the same path.
 2018-12-06 10:02:14 +01:00
Sybren A. Stüvel 041af5d3bb Bumped version to 0.6.1 2018-11-30 14:03:31 +01:00
Sybren A. Stüvel 5fa99981de blocks: Allow choosing number of DNA types shown via --limit CLI arg 2018-11-30 14:03:20 +01:00
Sybren A. Stüvel 595c02a07d Bumped version to 0.6 2018-11-30 13:59:33 +01:00
Sybren A. Stüvel 78981ce9f1 Added 'bat blocks' command that shows the DNA types that use the most data 
It's work in progress and maybe not useful for anyone but us.
 2018-11-30 13:59:20 +01:00
Sybren A. Stüvel b52faa6e75 Added Twine to dev requirements 
Twine is the recommended way to upload files to PyPi.
 2018-11-27 16:06:51 +01:00
Sybren A. Stüvel e898b678dc Typo 2018-11-27 16:00:50 +01:00
Sybren A. Stüvel fd1c04c752 Added instructions on how to upload packages to pypi 2018-11-27 15:59:58 +01:00
Sybren A. Stüvel 16521f5fc5 Removed logging options from setup.cfg again 
The % symbols confuse setup.py when running 'setup.py sdist bdist_wheel',
and I couldn't quickly figure out how to escape those properly.
 2018-11-27 15:59:43 +01:00
Sybren A. Stüvel 7b9312b684 Bumped version to 0.5 2018-11-27 15:51:19 +01:00
Sybren A. Stüvel 33512d42cf Multi-threaded compressed file transfers 
For regular file transfers (so to a directory, not to a ZIP file or S3
storage), use multi-threaded transfer when compressing. Compressing is
CPU-bound, so using multiple threads speeds things up considerably
(packing a Spring lighting file went from 6min30 single-threaded to
2min13 multi-threaded on my machine).
 2018-11-27 15:50:20 +01:00
Sybren A. Stüvel b6c0d01e45 bat pack --help improvements 2018-11-27 14:37:15 +01:00
Sybren A. Stüvel 521c7e1916 Added --compress option for 'bat pack' command 
This compresses all packed Blend files. Other files, as well as already-
compressed Blend files, are left as-is.
 2018-11-27 14:29:37 +01:00
Sybren A. Stüvel c1aaa3aab3 Add some log level configuration to py.test config 
Not sure if we want a global DEBUG setting; if it becomes annoying, change it.
 2018-11-27 14:29:37 +01:00
Sybren A. Stüvel 0b5e034a92 Fix MyPy errors when running without boto3 installed 2018-11-07 13:03:07 +01:00
Sybren A. Stüvel 43e8a7e370 Include version in --help output 2018-07-11 12:44:29 +02:00
Sybren A. Stüvel 439f727815 Bumped version to 0.5-dev 2018-07-10 16:03:30 +02:00
Sybren A. Stüvel 0e7411a42b Bumped version to 0.4 2018-07-10 16:03:00 +02:00
Sybren A. Stüvel d7b06f0644 Blender 2.8 compatibility of image datablock 2018-07-10 16:02:14 +02:00
Sybren A. Stüvel 504b50178b Moved from requirements.txt to pipenv for package management 2018-07-10 16:02:14 +02:00
Sybren A. Stüvel 38b3e2b55c Fixed a whole bunch of MyPy errors 
New release of mypy picks up more.
 2018-07-10 16:02:14 +02:00
Sybren A. Stüvel 893c1f2231 Convert hexified ID name to upper case 
Our test case accidentally has a numerical-only hex representation, which
doesn't show this issue.
 2018-07-05 11:53:38 +02:00
Sybren A. Stüvel 63118d250c Bumped version to 0.4-dev 2018-07-03 15:07:34 +02:00
Sybren A. Stüvel 8a308ef6e4 Bumped version to 0.3 2018-07-03 15:06:27 +02:00
Sybren A. Stüvel 0b96635953 Fix T55539: BAT doesn't pack particle caches 2018-07-03 15:05:15 +02:00
Sybren A. Stüvel aa8e2a8cdd Tracer fix for T55539 
This only fixes the tracer; the problem with the packer still exists.
 2018-07-03 15:05:15 +02:00
Sybren A. Stüvel f75f35ae77 Swap 'expected' and 'actual' arguments 
Python itself doesn't make any distinction (it just logs 'a != b') but
PyCharm does.
 2018-07-03 14:30:59 +02:00
Sybren A. Stüvel 26c5d2dbfc Unignore some paths 2018-07-03 14:00:53 +02:00
Sybren A. Stüvel d103056558 Flush the Packer callback queue when closing 2018-07-03 13:01:36 +02:00
Sybren A. Stüvel f1cfc28a1c Report skipped files 2018-07-03 13:00:10 +02:00
Sybren A. Stüvel 7fe06c247b Also report transfer progress on MOVE 
Previously transfers were only reported when copying files.
 2018-07-03 12:56:08 +02:00