Messages

Here is my current game plan for "named/character entity" substitution:

• Automatically substitute the following entities:
  
  •  = 
  • ©=©
  • ®=®
  • ™=™
  • –=–
  • —=—
  • ‘=‘
  • ’=’
  • “=“
  • ”=”
  • …=…
• For any named entity that is not mapped, display a space character. This should prevent truncating the content with a parsing error, which doesn't seem to me like a good user experience.
• If your books have any other named entities that you would like to render, you can add those to the tool by updating a file called entity.txt in the app shared storage folder. I'll include more details in the online documentation for how to accomplish that when I release the updates.

If you (or anyone else reading this thread) have suggestions for other entities that should be handled by default, let me know and I'll consider adding those as well.

I've already implemented the above, but I want to test it more thoroughly, along with some other updates that are already in progress. I think I'll be able to release in the next 2-3 weeks.

Karl

Yes, ePub is made up of XHTML files and not HTML, and XHTML doesn't support all named entities as far as I know. In fact I think it only really supports a small set of about 5 entities. It should support the decimal version of the entity though (e.g. — can be substituted with — - I've verified that works fine).

I think the level of support for named entities varies with the browser and what is defined in the DTD that is applicable for the document type. In the case of my app, I'm using the Android WebView component (based on Chromium, which is lower level component and not a full browser like Chrome). In any case, the error displayed by the WebView renderer is that the mdash name is not defined. I was able to add the definition to the document using an !ENTITY tag and that works as well, but that's not something I can do in my app dynamically (once the document is rendered I can't inject the definition any longer).

So if you have control over the ePub content, you can substitute the decimal version of the entity rather than using the named version.

In the mean time, I'm going to see if I can do that substitution automatically. I don't think it should be a problem, but I don't know what the impact to performance will be. So I'll see how that goes. If all goes well, I might only support a subset of what I think are the more common named entities that I think might be used.

I'll post again with more details once I figure things out better.

Karl

OK, I've done some exploring with multiple profiles and here's what I can make out:

• Most of my exploration was done in Android emulators, so it may not match exactly what you will experience in FireOS. FireOS 7.3.2.7 is supposed to be based on Android 9, however, so I suspect that the behaviour I see in the Android 9 emulator should match what you will see.
• Each profile has it's own "sandbox" area in primary storage that it has access to. In this way, each profile is isolated from others so that any customization (most anyway - there are some exceptions which I won't get into here) are unique to that profile and don't impact any other. That isolation includes all files visible to the profile, not just files private to individual apps.
• As I alluded to in my last post, secondary profiles don't have access to the SD card. This might be controlled by device manufacturers though, so may not be universally true. In the emulators (and in the physical devices I've tried), I had no access to SD card data from non-system applications in anything other than the primary profile.
• All of the above means that a secondary profile won't have access to book files that are stored in the primary profile "sandbox". In order to make book files visible to secondary profiles, you'll need to copy those files to the secondary profile sandbox.
• When you connect your device to a PC via USB, the PC exposes the primary storage area as a drive for the active user profile (and only one profile can be active at a time). So to load books into the secondary profile sandbox, you should switch to that profile (making it active) and then connect. The drive exposed on your PC will be the sandbox for that profile. If you copy book files there, they should be visible to the secondary profile on your device. Of course that means you need to have multiple copies of books files on your device, one for each profile, which means you'll eat up more storage.

On Android 9 (and presumably lower) there is a trick that works for me to share a single set of book files across profiles, thus avoiding the need to have duplicate copies of those files for each profile. It seems to be a hole in the profile sandbox implementation that Google left open up to Android 9, but appears to have been closed after that (in Android 10 the trick does not work for me). Whether the trick works for FireOS 7 or not I don't know, but since it is based on Android 9 it might. So I'll describe it here in case you want to try it:

• With the primary profile active, navigate to folder /Android/obb. If the folder does not exist, create it. You should be able to do this through the PC while connected through USB or on the device using a file manager app.
• Copy book files into this folder. You can create sub-folders to organize the files if you want.
• Switch to the secondary profile. In the secondary profile you probably won't see the same /Android/obb folder automatically (I didn't anyway), but you can create one. Once created, the folder seems to get linked to the same shared folder used by the primary profile so any book files you copied there while that profile was active should "magically" appear in the new /Android/obb folder you just created in the secondary profile.
• You should be able to scan that folder in my app now (after adding an appropriate root folder of course).

The obb folder seems to be the mechanism that older Android versions used for sharing files across profiles. Google appears to have disabled that (probably only for non-system app is my guess) as of Android 10, but it might still work prior to that.

Hopefully some of that information might help.

Karl

It's been a little while since I've tried using multiple profiles in Android, but from what I remember, I was only able to see the device SD card from the primary profile and not from any secondary profiles. That seemed (at the time at least) to be intentional behaviour in Android (for reasons that I didn't really understand - it doesn't make much sense to me to restrict the device that way).

I don't know if that is related to what you are seeing or not. If you're not using an SD card for book files then it wouldn't be. But I thought I'd mention it.

I'll try setting up profiles and see if I can get anything to break. I'll post back when I've had a chance to test it out.

Karl

Version 4.5.4 is now live on the Amazon App Store.

For anyone who has been out of commission over the last couple of weeks, I apologize. You should be able to update to get the fix now.

Karl

An update on this issue:

I've gotten my hands on a Fire (5th gen) with FireOS 5.7.0.0 and have been able to figure out what the issue is with version 4.5.2 of my app. Version 4.5.4 does in fact fix the issue (as confirmed above).

The issue should only happen if you have a Fire device on FireOS 5.x with a recent version of WebView (i.e. 100). If your Fire has FireOS 5.x with older WebView (i.e. version 59), or if your FireOS version is anything other than 5.x (either older or newer), then you should not have a problem with version 4.5.2. If you are experiencing the issue though, you will need version 4.5.4 of my app (or any version other than 4.5.2 - the issue is specific to that one).

Unfortunately, Amazon is still dragging its feet clearing up my Amazon App Store submission status for both Full and Basic versions of my app, so as of this morning, I am still unable to post the fix to that app store. I will let you know when I finally release 4.5.4 with the fix.

If you have access to other app stores like Google Play, or if you are OK with side-loading, I've posted details about those options above.

Sorry for how long this is dragging out, but I'm not sure what else I can do other than communicate with Amazon through public forums to try and get some focus on the issue - which I have been doing but with as of yet limited success.

Thanks for letting me know!

I've got a Fire (with FireOS 5.7.0.0) on order from eBay that should be coming soon. With that I hope to be able to figure out what happened with the previous version and hopefully find a workaround for it.

I'm happy the update fixed the problem in any case.

Unfortunately, Amazon is being ominously quiet on the tickets I've opened to clear up my eLibrary Manager and eLibrary Manager Basic Amazon App Store status so I can release my update. Again, I'll do that as soon as I can.

In the mean time, I've released version 4.5.4 on all other app stores (including Google Play). So if anyone is having issues with the last update on your Fire tablet and you have access to Google Play, you can get the update through that channel.

If you are using eLibrary Manager Basic and you are comfortable with sideloading, you can also get the .APK file at

• https://kpwsite.com/home/content/eLibrary/download.html

You can find a link for 4.5.4 towards the bottom of the page.

Hopefully Amazon will enable release of the update soon and I'll get it out on the Amazon App store as well. I will post here when that happens.

Sorry it's taking so long to post the update.

Amazon still hasn't cleared up whatever issue they are having on their end in the App Store with my previous submission (it's still in an "Approved" status and blocking updates). The messaging on the developer console has changed over the last couple of days though, so I hope that's a sign that someone is doing something. I'm keeping my fingers crossed.

Anyway, as soon as I can I will post that update.

I have my update prepared to release to the Amazon App Store which should fix the issue (I believe - but I won't know for sure until I get it out there). Unfortunately, both the Full and Basic apps seem to be stuck in an "Approved" status in the App Store (they should be "Live") and therefore I am unable to release the fix until the status changes. I've opened a problem ticket with Amazon so hopefully they will respond soon with a resolution.

I'll release the new version (hopefully with the issue fixed) as soon as I can get my previous submission status cleared up.

Sorry for the delay. It seems everything likes to go wrong at the same time.

I'll keep you posted.

Thanks for the replies. My best guess is that the webkit update caused this regression. I'm preparing another update (4.5.3) for both Basic and Full versions that revert back to the previous version of webkit. I'm also including logic to capture exception details in a log file if the issue continues. Once you get the update, if the problem persists (I'm hoping it won't), please send me a copy of the log file. When I post the update I'll add another comment here with more details about where to find the log file, should it be necessary.

I'm hoping to release tonight (it's 7:30AM local time for me now) after I'm done work (my day job). Updates show up quickest on Google Play, so if I get the update out tonight, you should be able to see it on Google Play soon after. I don't think 4.5.2 has gone live yet on the Amazon App Store as that version is still sitting in an "Approved" status rather than "Live". So hopefully this upcoming update will get out there in the Amazon ecosystem before 4.5.2 even goes live.

I'm assuming both of you received the update through Google Play, but if not, let me know so I know I don't miss anything.

Once I get things back up and running (hopefully soon), I'll see if I can find a FireOS 5.7 device to try and debug this better.

Did the problem just start happening with the latest update? I.e. was the previous version working properly? The latest version had updates to some Android libraries so it's possible that introduced an incompatibility with the WebView being used on that version of FireOS. Specifically, the webkit library was updated so that sounds like a likely suspect to have introduced incompatibilities.

Since this seems to be an Amazon specific version of WebView, I'm not sure I'll be able to get that installed on any of my devices. I'm adding some logging for this situation, so I'll put out an update soon which will capture a log file with more information when the failure happens. It might take me a few days to put that update out, but if you're willing to side load, I can give you the update a little sooner.

I'll also do some more investigation to see if the latest webkit library I'm using might have issues with the Amazon WebView component.

I don't have a Kindle Fire to test on, but I see that Fire OS 5.x is based on Android 5.1. I don't see this error on an emulator running Android 5.1.

This error happens when the WebView component fails to load (i.e it throws an error when the reader tries to start up). Can you let me know what version of WebView is loaded? On a vanilla Android device you would be able to see that through Settings/Apps/Android System WebView. I assume the same should be available on your Kindle Fire.

If I know what WebView version your device is using, I can try and see if I can load the same on my device and see if the same issue happens.

For the original poster and anyone else following this thread...

I can see two reasons for the behaviour reported:

• The scan gets into a loop, possibly due to recursion because of symbolic links.
• The scan "stalls" because it is blocked waiting for access to a file that is blocked through some OS level lock on a file.

For the first issue I've added a limit to the allowed recursion to prevent the scan from running indefinitely. For the second issue I've added logic to terminate a scan if any step waits for more than a minute (implying that there is something preventing the scan from continuing, possibly indefinitely). In both cases the final notification identifies the situation, also identifying the book file that was being loaded when the timeout occurred for the second case.

Hopefully that should cover the issue.

I'll release the updates soon after I've had a chance to test them more fully (as well as the other updates I've been working on recently).

Karl

It seems that the scan might be freezing when it tries to process a specific book. I'm not quite sure why that would happen. If you can pinpoint the book you can try and reload that book to your device from calibre. I think I may have run into something like that in the past as well and recopying the book fixed it. To stop the scan, try renaming the author folder temporarily and force stop the app. With the folder name changed, even if the scan starts again automatically it should end because the folder doesn't exist any more.

So what I would try is
1) Rename author folder (add a ".0" prefix or something).
2) Force stop the app.
3) Start the app again and wait for the scan to be auto-started (it should finish immediately since the folder doesn't exist any more).
4) Reload the books from calibre for that author onto your Kindle.
5) Rename the author folder back to it's original name.
6) Try scanning the folder again.

Hopefully the reloaded book files will work better.

Again, I'm not sure why a corrupted book would make the scan freeze like that. If I could get a sample of a book file that causes that behaviour for me as well, I should be able to figure out what is happening.