41 Commits

Author SHA1 Message Date
Sybren A. Stüvel
03fb4da583 Convert target path from Path to str & PurePath
The target path is just read as string from the CLI now, to allow more
complex targets (such as URLs) that don't directly map to a path.

The Packer subclass now handles the conversion from that string to a
`pathlib.PurePath`, and specific subclasses & transfer classes can convert
those to a `pathlib.Path` to perform actual filesystem operations when
necessary.
2019-02-26 16:35:08 +01:00
Sybren A. Stüvel
c4a0d5925a Use 'bat-' prefix when creating temporary directories 2019-02-26 16:35:08 +01:00
Sybren A. Stüvel
55a14eca1b Removed some extraneous commas 2019-02-04 15:17:14 +01:00
Sybren A. Stüvel
d222fea960 Explicitly write pack-info.txt as UTF-8 2019-01-02 15:14:50 +01:00
Sybren A. Stüvel
772a5a1b1c Replace → with ->
Apparently Windows still isn't using UTF-8 by default, and fails when
handling these log entries.
2019-01-02 15:14:50 +01:00
Sybren A. Stüvel
37ad8f8dfc Added --relative-only option to bat pack command
This makes BAT skip assets that are referred to with an absolute path.
It is assumed that the receiver of the BAT pack can access those assets
at the same path.
2018-12-06 10:02:14 +01:00
Sybren A. Stüvel
521c7e1916 Added --compress option for 'bat pack' command
This compresses all packed Blend files. Other files, as well as already-
compressed Blend files, are left as-is.
2018-11-27 14:29:37 +01:00
Sybren A. Stüvel
38b3e2b55c Fixed a whole bunch of MyPy errors
New release of mypy picks up more.
2018-07-10 16:02:14 +02:00
Sybren A. Stüvel
0b96635953 Fix T55539: BAT doesn't pack particle caches 2018-07-03 15:05:15 +02:00
Sybren A. Stüvel
d103056558 Flush the Packer callback queue when closing 2018-07-03 13:01:36 +02:00
Sybren A. Stüvel
fc144138d0 Better transfer error handling 2018-06-05 15:12:10 +02:00
Sybren A. Stüvel
85840faf70 Added docstring 2018-03-22 10:10:50 +01:00
Sybren A. Stüvel
e53470d898 Add a pack-info.txt file at the top of the BAT Pack
This file explains which blend file this pack was created for.
2018-03-21 16:33:41 +01:00
Sybren A. Stüvel
53e22a4a2f Added GPL comment blocks with proper attibution. 2018-03-16 13:25:20 +01:00
Sybren A. Stüvel
f7150c0d29 Allow aborting a pack operation.
For this to work well I also had to remove the sorting of blocks in
trace.deps(). The sorting caused the first `yield` to be executed only
after each blend file was opened, which means that the consuming for-loop
takes a long time to hit its first iteration. As a result, it would respond
slowly to abort requests. By not sorting the first `yield` is much sooner,
resolving this issue.
2018-03-16 12:12:15 +01:00
Sybren A. Stüvel
4f4a67784d Small cleanups
- No more shadowing of `blendfile` name.
- Removed unused import
- Formatting
2018-03-16 10:10:29 +01:00
Sybren A. Stüvel
889f3abd25 Implemented reporting callbacks.
Both the dependency Tracer class and the Packer class now support a
callback object, where the latter is a subclass of the former.

For file transfers running in a separate thread, there is a thread-safe
wrapper for progress callbacks. This wrapper can be called from any thread,
and calls the wrapped callback object from the main thread. This way the
callback implementation itself doesn't have to worry about threading
issues.
2018-03-15 17:59:55 +01:00
Sybren A. Stüvel
d2e13750e9 Moved comment to correct location 2018-03-15 16:28:34 +01:00
Sybren A. Stüvel
6b51183095 Packer: store the output path of the blend file in the target directory
This allows the Flamenco Add-on to get the final path of the blend file.
2018-03-14 16:06:21 +01:00
Sybren A. Stüvel
8340df129d BAT Pack can now exclude files based on glob patterns 2018-03-14 16:06:21 +01:00
Sybren A. Stüvel
0e9de7753c Allow Packer to be used as context manager + test for rewriting img seqs 2018-03-14 12:50:27 +01:00
Sybren A. Stüvel
c9664ca045 Report missing files in Packer.strategise() 2018-03-14 12:24:26 +01:00
Sybren A. Stüvel
575a0921c1 bat.pack.queued_copy → bat.pack.filesystem 2018-03-09 17:04:29 +01:00
Sybren A. Stüvel
15ca2f98ab Better handling of KeyboardInterrupt to abort file transfers 2018-03-09 16:30:46 +01:00
Sybren A. Stüvel
3ae39104a9 Allow subclasses of Packer to specify a different file transferer class 2018-03-09 14:25:43 +01:00
Sybren A. Stüvel
cc06a191a1 Added more annotations for mypy
Function declarations that have no type annotations at all will be skipped
by mypy. Adding an explicit `-> None` tells mypy to run on those functions
too.
2018-03-09 14:23:43 +01:00
Sybren A. Stüvel
38978537bc Removed some unused code + different line wrap style 2018-03-09 13:04:26 +01:00
Sybren A. Stüvel
ac3f6142b3 Move rewritten blend files instead of copying them
This will cause a nice speedup if the temporary directory happens to be on
the same file system as the target directory for the BAT Pack. This can be
influenced by setting an environment variable; see the documentation for
[tempfile.gettempdir](https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir)
2018-03-09 12:47:50 +01:00
Sybren A. Stüvel
824ca4bcb7 Rewrite blend files in temporary directory, before copying
This allows us to later support uploading to a non-local filesystem, such
as Amazon S3 cloud storage, after path rewriting is complete.
2018-03-09 12:33:15 +01:00
Sybren A. Stüvel
fdbbc3a20d Static type checking with mypy
This does introduce some not-so-nice things, like having to annotate each
`__init__` function with `-> None`. However, the benefits of having static
type checking in a complex bit of software like BAT outweigh the downsides.
2018-03-09 11:09:47 +01:00
Sybren A. Stüvel
632d01334c Renamed module tracer → trace
This is consistent with the other module names.
2018-03-09 10:02:44 +01:00
Sybren A. Stüvel
e8f41f2735 Store rewrites in AssetAction
This is in preparation of interchanging copying and rewriting.
2018-03-09 10:00:46 +01:00
Sybren A. Stüvel
d10264695a Tweaked logging 2018-03-08 15:30:31 +01:00
Sybren A. Stüvel
b6527ec1ee Moved file copy queue to separate module.
Also transformed into a threading.Thread subclass, rather than using a
function + module-global state variables.
2018-03-08 13:00:56 +01:00
Sybren A. Stüvel
08f538d8c8 File I/O improvements
- Tracer now iterates over blocks in disk order.
- Packer copies files per directory, in a separate thread.
- Packer only copies files if they don't exist yet.
- Packer also copies file permissions.
2018-03-08 12:33:15 +01:00
Sybren A. Stüvel
b5418e1f5f Removed Packer._packed_paths
The info in that dict is also available in Packer._actions[...].new_path
2018-03-08 11:15:38 +01:00
Sybren A. Stüvel
8fc1072144 Formatting 2018-03-08 11:08:57 +01:00
Sybren A. Stüvel
ccdccd69cf Speed up rewriting paths by reusing cached DNA structs & blocks 2018-03-08 10:59:45 +01:00
Sybren A. Stüvel
e2123f8090 Path rewriting for sequences seems to be working too. 2018-03-06 16:41:16 +01:00
Sybren A. Stüvel
71dd5bc11b Path rewriting when packing.
Doesn't work with sequences, nor with split dirname/basename fields.
2018-03-06 16:06:36 +01:00
Sybren A. Stüvel
433ad8f16a Moved packer code into separate class, in preparation for smarter behaviour 2018-03-06 11:58:38 +01:00