This repository was archived by the owner on Apr 29, 2020. It is now read-only.
fix: only output unixfs things #49
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a `unixfs-importer` but it's possible to output CIDs that resolve to `dag-raw` nodes if your data is small and you set `rawLeaves` and `reduceSingleLeafToSelf` to true. In that case you'll get a `dag-raw` node as the output. This makes it impossible to set metadata on small files with `rawLeaves` set to true. BREAKING CHANGE: If your data is below the chunk size, and you have `rawLeaves` and `reduceSingleLeafToSelf` set to true, you'll get a CID that resolves to a bona fide UnixFS file back with metadata and all that good stuff instead of a `dag-raw` node.
achingbrain
added a commit
that referenced
this pull request
Jan 16, 2020
Builds on #49. A separate PR as it's slightly contentious. Sets default DAG construction to be a `dag-pb` root node, `dag-pb` intermediate nodes and `ipld-raw` nodes for leaves. This will make parsing ever so slightly faster and DAG sizes ever so slightly smaller as there is no protobuf wrapper for the actual file data. Currently you may end up with `ipld-raw` leaves or `dag-pb` leaves that contain UnixFS entries with type 'file' or 'raw' depending on where the importer is invoked from. E.g. to generate the same CIDs as go-IPFS, `ipfs.add` will result in a balanced DAG with UnixFS leaf nodes with a type 'file', `ipfs.files.write` will result in a trickle DAG with UnixFS leaf nodes of a type `raw`, and specifying CID version 1 will get you `ipld-raw` leaf nodes and whatever tree strategy you specifed, default balanced. I think this is chaos, we should use `ipld-raw` leaf types everywhere and only offer options to change the DAG structure, not leaf types.
|
This will break interop with go-ipfs right? |
|
Yes, go-ipfs still outputs CIDs with the |
Closed
This was referenced Mar 1, 2020
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a
unixfs-importerbut it's possible to output CIDs that resolve todag-rawnodes if your data is small and you setrawLeavesandreduceSingleLeafToSelfto true. In that case you'll get adag-rawnode as the output.This makes it impossible to set metadata on small files with
rawLeavesset to true.BREAKING CHANGE:
If your data is below the chunk size, and you have
rawLeavesandreduceSingleLeafToSelfset to true, you'll get a CID that resolves to a bona fide UnixFS file back with metadata and all that good stuff instead of adag-rawnode.