Import functionality in Known

[srcr]

I would like to see a way you can import all posts in some form into Known.
In my opinion there is no need to make it feature rich, but a basic CVS import or similar would be great.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[rhiaro]

+1

[benwerd]

Planning on working on this intensively next week.

On Fri, Sep 26, 2014 at 12:43 PM, Amy Guy notifications@github.com wrote:

+1

—
Reply to this email directly or view it on GitHub
#480 (comment).

Ben Werdmuller
http://goog_1933028737
benwerd.com | werd.io

+1 (312) 488-9373

[mapkyca]

Similarly, Known to Known I/O would be handy - rich, with object/annotation transfer + file data export. Automatic translation of UUIDs etc.

[srcr]

I agree to that. I still want to migrate my data from mysql to mongo (or even better postgresql)

[mapkyca]

Probably the simplest would be to do a slightly more complete json output view with base64 encoded attachment blobs, reconstruction would be a matter of taking that and writing it into the new data structure - should be fairly uniform so artefacts should be minimal.

You'd end up with a big data structure mind...

[benwerd]

I'm examining both Twitter and Google's export formats right now. Comes down to a JSON version and a version that's HTML with embedded microformats. Known-to-Known communication is also a good idea - eg via an API address.

[benwerd]

To clarify, Twitter actually gives you a bunch of JSON files, an HTML-based browser interface, and a CSV. The latter isn't appropriate for us, but it's a useful thing to learn from.

[mapkyca]

If you're going that way you could output as XML/xslt - rock it like its 1998!

Seriously though, a static rendering of the machine readable version would be kind of cute.

[dealingwith]

+1 ...do you have a branch you're working on ATM that you can share?

[benwerd]

It's actually in trunk - still lots of work to do. But a static HTML + JSON
export should hopefully be ready to go by the end of the day.
On Oct 3, 2014 8:32 AM, "Daniel Miller" notifications@github.com wrote:

+1 ...do you have a branch you're working on ATM that you can share?

—
Reply to this email directly or view it on GitHub
#480 (comment).

[mapkyca]

Another suggestion, although it might be better as a plugin, it'd be cool if there was an import that accepted wordpress' export format.

Should be fairly straightforward....

[ipranjal]

@mapkyca how can we import wordpress styled post into known ? will it be a form of blog post ?

[mapkyca]

Well, the export XML seems to spit out everything - posts, pages, files (linked via url rather than embedded, so you'd need to fetch those references), categories, tags and comments.

You'd have to process the file and determine posts from other wp:items, which seems to have to be done via a heuristic rather than a subset in the XML (helpful :/ but I could be wrong - the file is very large and I've only looked at it for a few minutes).

But, assuming you can identify the post, you'd need to import that, import the author, and import any referenced resources. It'd be logical to import wp items as blog posts. Pages on the other hand don't have a clean mapping.

[benwerd]

FYI: we've successfully imported a few WordPress blogs now. More to come.

[mindscratch]

Glad to see this coming along, when I read "You can then import these into other Known sites, or process them using other software." on the export page I thought importing had already landed.

[michielbdejong]

@mindscratch see also #644 - may be an alternative if you're happy to work with the raw .sql files.

[benwerd]

As of right now, the export bundle includes the filesystem as Known stores it, as well as a full SQL entities export.

[michielbdejong]

Great!! When is this expected to be pushed to *.withknown.com instances?

[benwerd]

This week. I'll update this thread once the roll-out is complete.

[davidmead]

Does this mean I'll finally be able to get an export from davidmead.withknown.com and import it to blog.davidjohnmead.com?

[benwerd]

Yes. Sorry if we dropped the ball there. You'll be able to get it on demand.

[davidmead]

Great. Thanks Ben.

[socialreporter]

I have the same issue as davidmead - is it possible to export from mediablends.withknow.com (done that) and then import to my digital ocean site?

[michielbdejong]

@socialreporter Unfortunately it still requires a bit of fiddling, but I got it working in the end and documented how I did it.

[michielbdejong]

@socialreporter I would be happy to help you via irc if/when you try this, I'm michielbdejong in #indiewebcamp on freenode (usually online most of European daytime).

[socialreporter]

Thanks Michiel

On 25 February 2015 at 14:16, Michiel de Jong notifications@github.com
wrote:

@socialreporter https://github.com/socialreporter Unfortunately it
still requires a bit of fiddling, but I got it working in the end and
documented
#734 (comment) how I
did it.

—
Reply to this email directly or view it on GitHub
#480 (comment).

+447970621696
http://socialreporter.com

[benwerd]

The export function on the hosted site and in the latest codebase now produces the metadata SQL.

[michielbdejong]

Oh, great!

[pierreozoux]

Can we close this thread now?

[davidmead]

OK. I still cannot import my Wordpress XML using the upload feature in the latest build.

Also, I exported from my hosted Known site, and now gives me a slew of new files I don't know what to with.

Any help would be greatly appreciated.

[mapkyca]

Can close..?

[benwerd]

I briefly closed this, but no: import is not as functional as it needs to be, and unfortunately people are still reporting problems.

[mapkyca]

Folk might want to kick the tires on the latest github head, I just pushed a bunch of tweaks which would be good to have a bunch of different output thrown at.

[davidmead]

I tired to import my WP export .XML file and it doesn't work.

This was using the 0.9.9 build from http://withknown.com - I'll update with 0.9.9-a from you @mapkyca, but do you know if the Wordpress imports works now?

[mapkyca]

To certain values of "work". Import / Export is a hard problem, with many variables/quirks and such and such that can cause issues, and it's not a well trodden path.

[lindner]

Would it be possible to add integration tests? 1) Import Test - Add import file to repo. - Run importer. - Dump internal DB (mysqldump? something else?) - First time? save this as 'Golden' result. - Subsequent runs verify that the data is imported correctly against the golden DB 2) Export Test - Perform test operations (create posts, etc.) - Run Export, save Golden copy in repo. - Subsequent runs compare with golden copy.

…

On Wed, Jan 2, 2019 at 6:52 AM Marcus Povey ***@***.***> wrote: <#480>To certain values of "work". Import / Export is a hard problem, with many variables/quirks and such and such that can cause issues, and it's not a well trodden path. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#480 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAF5y55zqBCAcWT_w0V7Wc_ErC_-VpGNks5u_MeYgaJpZM4Cnkx_> .