Menu
What's New
By:
Filters

Python tools

Added csv output to the skill scanner so it can be used with entropiawiki and other tools.
 
Just had a read through the descriptions. That sounds super useful (and didn't know about the existence of the json file). I really like the before and after thing and nice to see an open source skill scanner that could be build upon/re-used if needed. Must have taken ages to do the log aggregator! Good effort!!
 
Super interested to learn about the json file. Is there any information or a link to it on the website, or is it a hidden Easter egg?

In particular, I'm curious about the manner in which it may be used. I assume the only reason for the file to exists would be for people to make automated requests to scrape the text of the file, as your program seems to be doing. Would it be possible to, and would MindArk allow, programmers to request an updated copy of the file every second or so, in order to create a complete record of a hunt/mine/craft (computing the before/after of every looting instance)?
 
TheOneOmega said:
Super interested to learn about the json file. Is there any information or a link to it on the website, or is it a hidden Easter egg?

In particular, I'm curious about the manner in which it may be used. I assume the only reason for the file to exists would be for people to make automated requests to scrape the text of the file, as your program seems to be doing. Would it be possible to, and would MindArk allow, programmers to request an updated copy of the file every second or so, in order to create a complete record of a hunt/mine/craft (computing the before/after of every looting instance)?
Click to expand...
I didn't check for sure, but I suspect it is being referenced/used by the javascript in the items table that you can see on your account on the website. I note that table has some dynamic functionality that allows re-ordering based on which column you click so suspect that is enabled by way of the page using the data in JSON format and then presenting that in table form. That's probably its only reason to exist.

As for the only reason for the table to exist, it's been there for over a decade and a half (although think it used to be static HTML). Generally having a comprehensive list of your items outside of the game is handy - especially when you "loose stuff" (what storage box on what planet was it?) and historically people used to copy and paste it into tools that would value the items using resources like entropiaauction (run off the expiring auction items feed that used to be online - not sure it still is... or where it is if it still exists) or PEbay. Not sure if either of the those two are still around... probably not.

So yes, while it isn't the only reason for the data to exist, the items list, in one shape or another has more or less always been used by community tools.

As for if it would be possible to programmatically access the JSON at regular intervals without the need for human intervention... that depends. The key blocker is the sign in process and the maths captcha. Those will be pretty awkward to get round however it might be that once you have signed in once, you can maybe maintain a session via a cookie. I've heard of people simply grabbing a cookie from their browser and reusing it in their program to get access to some sites but that by no means also means this would be possible in the case of Entropia.

As for hunting tracking, there was a log aggregator in the example programs in the GIT repository that is tracking stuff as it shows up in the log and the log is fair game, just a text file on your PC. There would be benefits of being able to get at the JSON without (or with only initial) human intervention though.
 
Last edited:
Yeah, the json file is used by the client side javascript on https://account.entropiauniverse.com/account/my-account/my-items/. That's the reason why it exists.

Regarding automatically garbing this from a script, there are a few caveats. Like @Wistrel said you need to login first (password, captcha and one-time-code) so the easiest way would be to log in normally using your browser and extract the session cookie.

Another "problem" is that you can't access "my items" too often, you will get blocked. You can see this by just refreshing the "my items" page a few times.
So every second will not be possible, but maybe every minute or so (not sure what the timeout is).
Personally I just get the items before and after each hunt.

The last issue is what MA would think of a script like that, they might think it's an abuse and ban you? I haven't asked them, probably good to do first.


My long term goal is that we would have a really good hunting tracker (like "Little Big Mining Log" for hunting :))
The easiest way is to use the log files as much as possible because they are easy to parse.
For loot it's only possible to get that output when team hunting though.
You can upvote this one so we can get loot logs also when solo hunting:

TT value in team loot

Would be nice to also show TT value of the loot in the team chat, not just the number of items. Would also be really nice to be able to get the loot chat when solo hunting!
www.planetcalypsoforum.com www.planetcalypsoforum.com
The other option is to OCR, I think LBML uses this a lot.
I made some initial test with Python and Tesseract and I think it would be possible to OCR the loot output from the top left corner.
 
I seem to remember there was an option to turn off loot window and have loot in chat instead... I've never done this but guessing it would maybe work like team loot does? Not sure... possibly though you still won't get the TT value.

Interesting on the "problem" I actually had the exact same thing happen to me the other night when I was having a cursory look at how plausible it would be to write an auto-login. I wasn't sure if it was a coincidence though as I was doing it quite late and wondered if something had gone into maintenance. I thought also maybe it was because I was accessing the page without a browser but it sounds like, from what you say, you can achieve the same effect by hitting refresh a few times in a browser. That does seem a little odd... I wander wha the threshold for "too many times" or "too fast" is... I mean if it's say only once every 5 mins then that would be good for your thoughts around hunting before and after comparison at least. Would make development tricky though as you'd want to be testing a lot while you got things working right.

Anyhow cheers for the info, good to know it wasn't just me.
 
First off: very nice looking set of tools.

Second: Automatic fetching of the json is easily achieved by writing a browser plugin, since they have access to all the information the browser has. This is not difficult. It does however pose a security threat if made publically available for that same reason, it has access to everything the browser has access to, and as such could be used for malicious intents.

Furthermore, automating the process more than you have done in your before and after app provides little to no value. It is sufficient to get the information quickly enough when repairing and/or converting shrapnel (Since with an automated fetch those things would require manual intervention anyways, e.g. stop hunting for the period of time that it takes for your script to fetch new data, then convert/repair and once again wait for new data fetch before you can start hunting again).

So in short, automating the process is not worth it, because what you gain in reduced manual intervention is lost in time. In my opinion.

We do however need improved logging in Entropia. With proper logging of what costs are incurred (e.g. tool X inflicted Y damage) (You took Z points of damage to K body part) As well as loot logging we could have some very nice tracking tools available that requires minimal manual intervetion aside from setup.
 
Yeh have to say I concur... I concluded it wasn't really worth the effort too.
 
Updated "aggregate-log" script to include loot (from the new loot logs).

Also started on a new script "analyze-loot" to capture individual loot events and plot them.
It's very basic at the moment but can be seen as a basis to build more interesting stuff.
This is a plot from the script:

index.php


It's a Maffoid hunt with 589 kills in total.
 
Have added bits and bobs over time to https://github.com/opentropia/pytropia

Some notable things are that item-diff have nicer output and can take a file with MU to give output like:

Code: 
$ ./item-diff.py -m ~/Dropbox/eu/my-mu.json -a ~/Dropbox/eu/hunts/1135/start.json  -b ~/Dropbox/eu/hunt
s/1135/end.json
Expenses                                        PED      Count    Markup
-----------------------------------------  --------  ---------  --------
ArMatrix B-Amplifier 10P (L)                   0.00          0      0.00
Mayhem Easter Egg                              0.00         -1      0.00
Vehicle RK-0 (L)                              -0.00          0      0.00
ArMatrix BP-20, FEN Edition                   -0.01          0      0.00
...
Vigilante Harness, Adjusted (F)               -0.45          0      0.00
Armor Plating Mk. 5B                          -0.75          0      0.00
Mayhem B-Amplifier Delta (L)                  -0.79          0      0.00
Longreach 4                                   -3.49          0      0.00
Bullseye 8                                    -9.26          0      0.00
Universal Ammo                             -1751.04  -17510446      0.00
[Sum]                                       1767.26          0      0.01

Loot                              PED     Count    Markup
----------------------------  -------  --------  --------
Shrapnel                      1017.36  10173629     10.17
Animal Liver Oil               518.00       518     23.31
Soft Hide                       55.50       555      0.00
Primordial Longu Skin           18.75        25      0.00
Isis LLP 4 (L)                  12.50         1      0.00
Tier 7 Component                11.90        17     41.65
Output Amplifier Component       4.40        11      0.88
...
Atrox Fang                       0.02         1      0.00
Papplon                          0.00        56      0.00
Nissit                           0.00        24      0.00
[Sum]                         1644.01         0     76.01

Markup
  Gained    Lost    Delta    Delta (%)
--------  ------  -------  -----------
   76.01    0.01    76.01         4.30

Summary
   Cost    Return    Delta    Delta (%)    Delta MU    Delta MU (%)
-------  --------  -------  -----------  ----------  --------------
1767.26   1644.01  -123.25        93.03      -47.24           97.33

Also update analyze-loot to be able to produce some more fancy plots and also extract bonus shrapnel as a separate item.

loot-sim have been added to simulate a "hunt" based on cost to kill, efficiency and looter level: https://github.com/opentropia/pytropia/blob/master/loot-sim.py
The multipliers and probabilities:
Code: 
    # Tuned for 93% TT from normal loot + 6% from bonus shrap. 99% total
    loot_probabilities  = [0.100, 0.100, 0.250, 0.350, 0.120, 0.040, 0.020, 0.01, 0.0025, 0.0035, 0.002, 0.0018733, 0.00006, 0.00004, 0.00002, 0.0000067]
    loot_multipliers    = [0.282, 0.376, 0.469, 0.525, 0.941, 1.883, 3.765, 7.53, 11.275, 15.045, 18.87, 33.5     , 150    , 250    , 700    , 1500]
are based on real data (around 200K individually tracked kills) and up to the 20x I think it's pretty accurate. To get accurate probabilities for 100x and above a lot more kills needs to be tracked though.
 
miathan said:
Updated "aggregate-log" script to include loot (from the new loot logs).

Also started on a new script "analyze-loot" to capture individual loot events and plot them.
It's very basic at the moment but can be seen as a basis to build more interesting stuff.
This is a plot from the script:

index.php


It's a Maffoid hunt with 589 kills in total.
Click to expand...

This and the stuff after look really nice. What did you use to do the graphing out of interest? Never tried graph production in python before.
 
Not sure it's of interest but did this the other day. Just a minor thing to display expiring auction items in a terminal window. Not sure if it has much use.

www.entropiaplanets.com

Bob the Builder - Shut Down (2020-09-02)

I searched the forum but could only see 2 short threads about the site and both other 10 years old so figured I'd start a new one. I can remember...
www.entropiaplanets.com www.entropiaplanets.com

It's just a one liner if anyone wants the code
 
You must log in or register to reply here.
Back
Top