Developer PSA: labs.j-novel.club is your new friend!

redmasq

Just curious, when sign-in is implemented, are you planning it to be credentials only, or will we need to get an api key, client cert, or such?

chocolatkey

@redmasq There's really no point in restricting it in that manner, which can simply be circumvented by using a non-browser client or setting up a proxy to the API. More work for me, more work for you, zero sum game

redmasq

@chocolatkey I only ask since such extra measures have been en vogue as of late. I was thinking of doing a plugin for my copy of Calibre to get updates to ebooks and such.

chocolatkey

Added https://labs.j-novel.club/app/v1/auth/login and https://labs.j-novel.club/app/v1/auth/logout

Login payload (in JSON) is {"login":"<email or username>","password":"<password>"}. Optionally, slim: true can be added if all you need is a token, and don't need cookies.
400 is your fault, 404 means user not found, 401 means bad pass, 429 means too many tries, else assume a temporary outage or bug

Logout is just POST with valid auth to delete session associated with token and expire the cookies.

Icerius

@chocolatkey I would recommend returning 401 for both user not found and invalid password.

It is considered best practice to return the same result during authentication irrespective of if the error is invalid user or invalid pass

See chapter "Authentication and Error messages" in https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html for more info

chocolatkey

@icerius that's a fair point, it's changed now. There are still technically ways to enumerate users, thought I won't mention them.

redmasq

@chocolatkey Thanks for the hard work.

Concerning the username scraping, with that vector plugged up, the easiest way, if reading above correctly, would be once logged in, using rainbow tables, but a moot point since I'm fairly certain someone with a valid username can harvest from these forums anyways, in theory. Logging mixed with governor limits per login identity and per IP/subnet should mitigate much of that style. At least for stuff that I have maintained, an expiring minutely and a rolling daily counter set usually suffices. That selection allowed me to manage the minutely collection in-memory and queue up daily to allow atomic transactions for log and aggregate counts.

Regardless, hoping to setup keeping my local copies auto synced, so might code up a cron job this weekend. I may be willing to share a wrapper of whatever language I use if there is interest. Node.js (JavaScript), Python, or Powershell will be my most likely one of my choices, provided I don't get assigned honey-dos (not that this sort of thing is much more than http request and deserialization, but wouldn't be the first time).

chocolatkey

This post is deleted!

Crimson Wise

I am looking for some dogfooding

For reference, which kind of feedback are you looking for?

jhkghl678690iop

One thing is in order to get all the parts of the serie, it is possible to call:

GET https://labs.j-novel.club/app/v1/parts/{ID or slug}/toc

But the ID of a part of the series needs to be obtained first. Wouldn't it make more sense to have instead something like?

GET https://labs.j-novel.club/app/v1/series/{ID or slug}/parts

(ie using the series ID)

chocolatkey

@jhkghl678690iop I can add that. It was originally constructed this way for the app's sake

Malloc

I am having a hard time getting part data for a part that requires me to be logged in to download. On the old api, I would set up a header on the html GET request with a header called "Authorization" with a value of the sessionID I got from the login.

However, when I use the new API: "https://labs.j-novel.club/embed/{part ID}/data.xml" it always returns forbidden when I pass the session id in the Authorization header. I have the both the session id (from the old login API) and access cookie I got from the "set-cookie" header on the login response. However I don't know what header to put it in with the new api.

I have tried to figure out what headers to use however when downloading a part using your websites reader the headers appear to be hidden on the HTML request when using Chrome's developer tools but it appears the headers on the data.xml are hidden because it is embeded into a blob:https://labs.j-novel.club/XXXXX call

Crimson Wise

@malloc

I'm doing the following (replace XXX, YYY and ZZZ as required) and it works:

$ curl -X POST 'https://api.j-novel.club/api/users/login' -H 'Content-Type: application/json' -d '{"email":"XXX", "password":"YYY"}' -s
  {"id":"ZZZ","ttl":1209600,...}

$ curl 'https://labs.j-novel.club/embed/61d760a893e1a3e2123d0fb5/data.xhtml' -H 'Authorization: Bearer ZZZ' -s | head -n 6
  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE html>
  <html xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" xmlns:xml="http://www.w3.org/XML/1998/namespace" lang="en" xml:lang="en">
  <head>
    <meta content="text/html; charset=UTF-8" http-equiv="content-type"/>
    <title>Guide to the Perfect Otaku Girlfriend: Roomies and Romance Volume 5 Part 2</title>

You can also use instead the token from the new login endpoint and also works.

Malloc

@crimson-wise

Sweet. It was the additional "Bearer " I was missing in the authorization header. Thank you. I read about that in the first post so I thought they might have meant the authorization header was called "Bearer" and not that the authorization header was still authorization it just wanted a bearer in front of the token.

Thanks so much! I am gonna start working on replacing all of my old api calls with the new api ones one step at a time.

Malloc

said in Developer PSA: labs.j-novel.club is your new friend!:

Endpoints return protobuf-encoded responses by default. If you want to get JSON, add format=json to the query. If you want a text dump format, format=text.

Does anyone know where you put format=json? My application already deserializes json, and I was wondering how to request labs.j-novel.club to pass back the response in json.

I tried having a header format with a value of json and that still returns protobuf so I am obviously missing where I pass this in the query.

chocolatkey

@malloc it's a query parameter as it says. So in the URL query, e.g. /endpoint?format=json

Malloc

@chocolatkey Thanks again. I am relatively new to web development like this so you have been very helpful.

chocolatkey

@crimson-wise Don't forget you can use "slim": true in the login since you just need the token.

tim_pkmn89

I found out you guys had an API a few days ago, and thought I'd make use of it to play around with some new Python features.

A few questions/comments:

Is there a way to find out if a volume is still in the middle of releasing parts? Right now I'm just checking to see if the next volume exists. I don't want to make any assumptions based on the number of parts.
I noticed that the datetimes are different for the manga API calls. Specifically, the volume "publishing" field and the parts "expiration"/"launch" fields don't include milliseconds while everything else does. I had to add in some extra code to parse them out.
Ascendance of a Bookworm itself, V1, and V1P1 all share the same slug of "ascendance-of-a-bookworm". Is this known? It doesn't seem to be like that for any other title.

myskaros

@tim_pkmn89 said in Developer PSA: labs.j-novel.club is your new friend!:

Ascendance of a Bookworm itself, V1, and V1P1 all share the same slug of "ascendance-of-a-bookworm". Is this known? It doesn't seem to be like that for any other title.

I can answer this one; this is simply due to the way the original backend was programmed, where the slug is automatically generated based on the plain text title entered. Even if the title is changed afterward, the slug stays the same presumably since that slug is entered manually in other parts of the backend and it would be a pain to have to update all those fields if the slug were to change.