Books not found with scan

[mac1299]

I have several hundred books that are on my SD that are not found by scan.  Is there anything I can do?  Should I reinstall the missing books and try again. 

[kweckwor]

Some basic things to keep in mind (I suspect your situation is probably not one of these, but I'll suggest them just in case):

• The tool will only recognize ePub books by default.
• You need to grant access to folders where books you are trying to load reside on your device. You do that by adding "root folders" in the permission tab of the eLibrary Settings. That's assuming your device has an Android version no older than 7.0.

I assume the books you're trying to access are epub books (with .epub file extension). If you are trying to load other types of books, then you will need to enable that through the eLibrary settings (and getting access to metadata will also take some additional setup). I'll assume you're trying to load epub though unless you tell me otherwise.

If you have granted access to the locations where the books reside but the tool still can't see them, it's possible it's having issues with the book file content. If that's the case, you can send me a sample book that is not being found and I can try it out to see if I have any luck. If loading the books fails for me as well then I should be able to figure out what the issue is.

Karl

[mac1299]

I would say 95% of the books are Epub's.  A few MOBI's may be in there though.  I convert them all on Calibre first before I install them.  It seems that when I start a scan it never stops scanning.  Every time I try again it says that a scan is already in progress.  This could be 10 hours later.  Is there any way to stop a scan once its under way?  I have a Kindle Fire 8HD.  I have all my books on a 128GB SD disc.  Thank You for responding. 

[kweckwor]

It sounds like your scan may have gotten into an endless loop. I try to detect recursion to prevent that, but I ran into a scenario recently where I still get into an endless loop while testing a new feature I'm currently working on. I'm guessing there might be some symbolic links that may end up getting the scan into a repeating pattern that gets around the checks I have in place (or something like that - I haven't quite worked out the details of how I was able to get to that point yet).

In any case, I've added a cap on how many folders a scan will consider before stopping, suggesting that you narrow down the scan. That may get your scan to end. Unfortunately, I'm not quite ready to release that version just yet (I think I'll probably get it out in a couple of weeks). If you're comfortable with side-loading apks and you'd like to try this new version early, I can send you the apk with that limit implemented (the version is working generally but I need to test it some more and there are some final touches I'm making to it - so it's not quite finished just yet, but it should be good enough for use). Let me know if you're interested and I can send you a link where you can download it.

Unfortunately. with the version you've got now, I don't think there is a good way to stop the scan if it has indeed gotten into an endless loop. You can try force stopping the app, but since the scan uses Android's work manager, it will probably start again. Maybe if you're able to get a different scan started (a more targeted scan) prior to the previous scan being restarted, that might prevent the bad scan from starting again.

What is the root folder you're using for the scan? In the case where I ran into my issues, it was on an older version of Android where I'm able to scan from the root folder. That was the situation where I get into a seemingly endless loop. I'm not sure what version of Android the Kindle OS for the 8HD is based on. Do you need to add root folders for your device, or do you have the option to select root as the starting point (that appears as "Device" in the scan dialog)?

[mac1299]

I keep all my downloads in a "Books" folder.  They are all organized by author within this folder.  My Fire is OS 5.6.8.0.  I think what I will try next is to scan each subfolder(author) and not the entire book folder.  Only way to stop the scan is to uninstall elibrary and reinstall. 

[mac1299]

Ok I just began scanning each individual author and it froze during one of the scans.  They were all EPUB's under this author.  I'm really not sure what to do now.  So there is no way to stop the scan once its started? 

[kweckwor]

It seems that the scan might be freezing when it tries to process a specific book. I'm not quite sure why that would happen. If you can pinpoint the book you can try and reload that book to your device from calibre. I think I may have run into something like that in the past as well and recopying the book fixed it. To stop the scan, try renaming the author folder temporarily and force stop the app. With the folder name changed, even if the scan starts again automatically it should end because the folder doesn't exist any more.

So what I would try is
1) Rename author folder (add a ".0" prefix or something).
2) Force stop the app.
3) Start the app again and wait for the scan to be auto-started (it should finish immediately since the folder doesn't exist any more).
4) Reload the books from calibre for that author onto your Kindle.
5) Rename the author folder back to it's original name.
6) Try scanning the folder again.

Hopefully the reloaded book files will work better.

Again, I'm not sure why a corrupted book would make the scan freeze like that. If I could get a sample of a book file that causes that behaviour for me as well, I should be able to figure out what is happening.

[kweckwor]

For the original poster and anyone else following this thread...

I can see two reasons for the behaviour reported:

• The scan gets into a loop, possibly due to recursion because of symbolic links.
• The scan "stalls" because it is blocked waiting for access to a file that is blocked through some OS level lock on a file.

For the first issue I've added a limit to the allowed recursion to prevent the scan from running indefinitely. For the second issue I've added logic to terminate a scan if any step waits for more than a minute (implying that there is something preventing the scan from continuing, possibly indefinitely). In both cases the final notification identifies the situation, also identifying the book file that was being loaded when the timeout occurred for the second case.

Hopefully that should cover the issue.

I'll release the updates soon after I've had a chance to test them more fully (as well as the other updates I've been working on recently).

Karl

[LDad]

I'd like to add a note on this thread, since I assume it's related.
I can find the books I've loaded on my profile, however I have the application on my kid's, and it refuses to see any books.
I've tried the following solutions myself:
I have tried to export (not successful, but I figure that's supposed to be for an additional devise anyhow)
I've tried to add a sub-folder (unsuccessful through the PC, the App can see it, but it doesn't find the books within it)
I added a root folder in app (unsuccessful, and the PC can't see the folder to include material).
I had run this app successfully on a child profile on an older Fire, and it was great because I could isolate the internet and my kid would be reading books rather than webbing and gaming and all over the place. I'd really like to be able to continue to do that!
my elibrary build is 5.0.1/162/54
my Fire is an 8th gen running 7.3.2.7
Thank you!

[kweckwor]

It's been a little while since I've tried using multiple profiles in Android, but from what I remember, I was only able to see the device SD card from the primary profile and not from any secondary profiles. That seemed (at the time at least) to be intentional behaviour in Android (for reasons that I didn't really understand - it doesn't make much sense to me to restrict the device that way).

I don't know if that is related to what you are seeing or not. If you're not using an SD card for book files then it wouldn't be. But I thought I'd mention it.

I'll try setting up profiles and see if I can get anything to break. I'll post back when I've had a chance to test it out.

Karl

[LDad]

Thanks!
In our case, no, not using an SD card, just the internal storage with roughly 500 books.
On the main profile I've had very few issues, and no trouble with the inventory showing up on a search, it's jus the kid profile that can't view the material. No issues opening the app, just viewing the books.

[kweckwor]

OK, I've done some exploring with multiple profiles and here's what I can make out:

• Most of my exploration was done in Android emulators, so it may not match exactly what you will experience in FireOS. FireOS 7.3.2.7 is supposed to be based on Android 9, however, so I suspect that the behaviour I see in the Android 9 emulator should match what you will see.
• Each profile has it's own "sandbox" area in primary storage that it has access to. In this way, each profile is isolated from others so that any customization (most anyway - there are some exceptions which I won't get into here) are unique to that profile and don't impact any other. That isolation includes all files visible to the profile, not just files private to individual apps.
• As I alluded to in my last post, secondary profiles don't have access to the SD card. This might be controlled by device manufacturers though, so may not be universally true. In the emulators (and in the physical devices I've tried), I had no access to SD card data from non-system applications in anything other than the primary profile.
• All of the above means that a secondary profile won't have access to book files that are stored in the primary profile "sandbox". In order to make book files visible to secondary profiles, you'll need to copy those files to the secondary profile sandbox.
• When you connect your device to a PC via USB, the PC exposes the primary storage area as a drive for the active user profile (and only one profile can be active at a time). So to load books into the secondary profile sandbox, you should switch to that profile (making it active) and then connect. The drive exposed on your PC will be the sandbox for that profile. If you copy book files there, they should be visible to the secondary profile on your device. Of course that means you need to have multiple copies of books files on your device, one for each profile, which means you'll eat up more storage.

On Android 9 (and presumably lower) there is a trick that works for me to share a single set of book files across profiles, thus avoiding the need to have duplicate copies of those files for each profile. It seems to be a hole in the profile sandbox implementation that Google left open up to Android 9, but appears to have been closed after that (in Android 10 the trick does not work for me). Whether the trick works for FireOS 7 or not I don't know, but since it is based on Android 9 it might. So I'll describe it here in case you want to try it:

• With the primary profile active, navigate to folder /Android/obb. If the folder does not exist, create it. You should be able to do this through the PC while connected through USB or on the device using a file manager app.
• Copy book files into this folder. You can create sub-folders to organize the files if you want.
• Switch to the secondary profile. In the secondary profile you probably won't see the same /Android/obb folder automatically (I didn't anyway), but you can create one. Once created, the folder seems to get linked to the same shared folder used by the primary profile so any book files you copied there while that profile was active should "magically" appear in the new /Android/obb folder you just created in the secondary profile.
• You should be able to scan that folder in my app now (after adding an appropriate root folder of course).

The obb folder seems to be the mechanism that older Android versions used for sharing files across profiles. Google appears to have disabled that (probably only for non-system app is my guess) as of Android 10, but it might still work prior to that.

Hopefully some of that information might help.

Karl

[LDad]

Thank you!

The solution with this version of Android is a modification of your obb fix.
The folder exists within the "master" profile on the device, however it is invisible to the "child" profile.
Loading the books from the PC into the main obb folder works however there is a blockage in the permissions within the Fire OS to prevent the "child" profile from accessing USB connection control, so the only way to create the folder for that user is within the eLibrary software, within the Android file structure, but it snaps right into shape at that point.

Thanks much!