Hacking Python script help wanted (synology spk)

[Rasscal]

I have created a synology spk to run a couple of scripts from my synology NAS rather than hopping onto my desktop & command line

1st script & issue

Im having trouble working out the correct python formats for inserting a custom backup path within Sc0rpion's FTP_DB_Backup-50X script

https://github.com/Sc0rpion/FTP_DB_Backup-50X/blob/master/README.md

My aim is to have the PS4 backed up to my NAS via FTP, ive tried editing the below line within his script but i keep receiving an error saying path cannot be found. With my limited knowledge of python i dont know if my format is totally wrong or its becuase i have a space in my path or a combination of both.

https://github.com/Zer0xFF/PS4_db_rebuilder/blob/master/README.md

parser = argparse.ArgumentParser()
parser.add_argument("PS4_IP", help="PS4 ftp ip address")
args = parser.parse_args()
app_db = "tmp/app.db"
PS4_IP = args.PS4_IP
with many different variants im still presented with the below error

usage: fix_db.py [-h] PS4_IP
fix_db.py: error: the following arguments are required: PS4_IP


does anyone have any python knowledge that could help me out please.

 

[KiiWii]

Looks like you need to specify the IP of the PS4.

Are you adding IP : port or just IP?

 

[Rasscal]

wow fast reply

ive tried alsorts haha

Spoiler: EXAMPLES

parser = argparse.ArgumentParser()
parser.add_argument("192.168.0.108", help="PS4 ftp ip address")
args = parser.parse_args()
app_db = "tmp/app.db"
PS4_IP = args.PS4_IP


parser = argparse.ArgumentParser()
parser.add_argument("PS4_IP", help="192.168.0.108")
args = parser.parse_args()
app_db = "tmp/app.db"
PS4_IP = args.PS4_IP


parser = argparse.ArgumentParser()
parser.add_argument("192.168.0.108", help="1337")
args = parser.parse_args()
app_db = "tmp/app.db"
PS4_IP = args.PS4_IP


parser = argparse.ArgumentParser()
parser.add_argument("PS4_IP", help="PS4 ftp ip address")
args = parser.parse_args()
app_db = "tmp/app.db"
192.168.0.108 = args.PS4_IP



PS4_IP = 192.168.0.108
parser = argparse.ArgumentParser()
parser.add_argument("PS4_IP", help="PS4 ftp ip address")
args = parser.parse_args()
app_db = "tmp/app.db"
PS4_IP = args.PS4_IP


PS4_IP = 192.168.0.108:1337
parser = argparse.ArgumentParser()
parser.add_argument("PS4_IP", help="PS4 ftp ip address")
args = parser.parse_args()
app_db = "tmp/app.db"
PS4_IP = args.PS4_IP

parser = argparse.ArgumentParser()
parser.add_argument("192.168.0.108:1337", help="PS4 ftp ip address")
args = parser.parse_args()
app_db = "tmp/app.db"
PS4_IP = args.PS4_IP

becuase im trying to create a synology package its time consuming keep trying a configuration & installing/uninstalling the package & re-editing so i installed python for windows & running the modlue from python editor, also becuase of the time im doing my testing in a VM (while im at work haha) which means the PS4 isn't actually available to connect but if this was the issue the error i recieve would be different right?

 

[Sparkss]

Is that your entire script pasted above? The error sounds like you have a required variable defined as an input parameter, this is usually in the header of a script/or at least the first couple of lines. I know PowerShell uses these, I can't remember about Python as it has been a while since I did any serious scripting in that language. But the whole script will help.

secondly, you show the error, but not the command you are using to call the script. The full command, including the IP address, it should be a local IP (10. or 192.) so no issue with posting it that I can think of.

Both of these are needed to properly troubleshoot the script (in any language).

 

[Rasscal]

thanks

Issue 1

As stated in OP i can insert my IP in this one no problem but i dont know the correct format for my NAS share to backup too as my below example does not work.

full path is
volume3/N.A.S 2/PS4/Backup

original below in spolier

ive tried
pathofbackup=`dirname $0`/volume3/N.A.S 2/PS4/Backup

Spoiler: Full Script

#!/bin/bash
#

echo "Starting FTP BackUp."

pathofbackup=`dirname $0`/BackUps
ip_ps4="192.168.1.108"
nowdat=$(date "+%m_%d_%Y_%H.%M.%S")

download_path() {
local url=${1}
printf "Downloading ${url}: "
echo -n " "
wget -r --no-parent -P ${pathofbackup}/${nowdat} --progress=dot ${url} 2>&1 | grep --line-buffered "%" | sed -u -e "s,\.,,g" | awk '{printf("\b\b\b\b%4s", $2)}'
echo -ne "\b\b\b\b"
echo "DONE"
}

download_file() {
local url=${1}
printf "Downloading ${url}: "
echo -n " "
wget -P ${pathofbackup}/${nowdat}/DB_Dackup --progress=dot ${url} 2>&1 | grep --line-buffered "%" | sed -u -e "s,\.,,g" | awk '{printf("\b\b\b\b%4s", $2)}'
echo -ne "\b\b\b\b"
echo " DONE"
}

download_file "ftp://${ip_ps4}:1337/system_data/priv/mms/app.db"
download_file "ftp://${ip_ps4}:1337/system_data/priv/mms/addcont.db"
download_file "ftp://${ip_ps4}:1337/system_data/priv/mms/av_content_bg.db"

download_path "ftp://${ip_ps4}:1337/system_data/savedata"
download_path "ftp://${ip_ps4}:1337/user/home"
download_path "ftp://${ip_ps4}:1337/user/trophy"
download_path "ftp://${ip_ps4}:1337/user/license"
download_path "ftp://${ip_ps4}:1337/user/settings"
download_path "ftp://${ip_ps4}:1337/user/system/webkit/secure"
download_path "ftp://${ip_ps4}:1337/user/system/webkit/webbrowser"
download_path "ftp://${ip_ps4}:1337/system_data/priv/home"
download_path "ftp://${ip_ps4}:1337/system_data/priv/license"
download_path "ftp://${ip_ps4}:1337/system_data/priv/activation"

mv ${pathofbackup}/${nowdat}/$ip_ps4:1337 ${pathofbackup}/${nowdat}/UserData

size_this=$(du -sh ${pathofbackup}/${nowdat}| cut -f -1)
size_all=$(du -sh ${pathofbackup} | cut -f -1)

echo "FTP BackUp Complete. Developed by Sc0rpion"
echo "BackUp folder ${pathofbackup}/${nowdat} size = ${size_this}, size all BackUps = ${size_all}"

Issue 2

To run this one from my desktop cmdline would be

cd <path of script>
fix_db.py 192.168.0.108

but becuase i want to run this from a NAS spk the app will load a script on command so i wanted to incoriparate the PS4 IP address within the same script (if that makes sense )

Spoiler: Full Script

#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import print_function
from ftplib import FTP
import sqlite3
import appinfo
import io
import os
from sfo.sfo import SfoFile as SfoFile
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("PS4_IP", help="PS4 ftp ip address")
args = parser.parse_args()
app_db = "tmp/app.db"
PS4_IP = args.PS4_IP
if not os.path.exists('tmp'):
os.makedirs('tmp')
class CUSA :
sfo = None
size = 10000000
is_usable = False
info = {}
files = []
def sort_files(file) :
if("CUSA" in file) :
files.append("'%s'" % file[-9:])
def get_game_info_by_id(GameID) :
if(GameID not in info) :
info[GameID] = CUSA()
buffer = io.BytesIO()
ftp.cwd('/system_data/priv/appmeta/%s/' % GameID)
ftp.retrbinary("RETR param.sfo" , buffer.write)
buffer.seek(0)
sfo = SfoFile.from_reader(buffer)
info[GameID].sfo = sfo
info[GameID].size = ftp.size("/user/app/%s/app.pkg" % GameID)
info[GameID].is_usable = True
return info[GameID]
ftp = FTP()
ftp.connect(PS4_IP, 1337, timeout=30)
ftp.login(user='username', passwd = 'password')
if(len(files) == 0) :
ftp.cwd('/user/app/')
ftp.dir(sort_files)
print(files)
ftp.cwd('/system_data/priv/mms/')
lf = open(app_db, "wb")
ftp.retrbinary("RETR app.db" , lf.write)
lf.close()
conn = sqlite3.connect(app_db)
cursor = conn.cursor()
cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name LIKE 'tbl_appbrowse_%%';")
tables = cursor.fetchall()
files_joined = "SELECT %s AS titleid " % ' AS titleid UNION SELECT '.join(files)
tbl_appbrowse = []
for tbl in tables :
tbl_appbrowse.append(tbl[0])
print("Processing table: %s" % tbl[0])
cursor.execute("SELECT T.titleid FROM (%s) T WHERE T.titleid NOT IN (SELECT titleid FROM %s);" % (files_joined, tbl[0]))
list_id = cursor.fetchall()
sql_list = []
for tmp_GameID in list_id :
GameID = tmp_GameID[0].replace("'", "")
print(" Processing GameID: %s... " % GameID, end='')
cusa = get_game_info_by_id(GameID)
if(cusa.is_usable == True) :
sql_list.append("""("%s", "%s", "%s", "/user/appmeta/%s", "2018-07-27 15:06:46.822", "0", "0", "5", "1", "100", "0", "151", "5", "1", "gd", "0", "0", "0", "0", NULL, NULL, NULL, "%d", "2018-07-27 15:06:46.802", "0", "game", NULL, "0", "0", NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, "0", NULL, NULL, NULL, NULL, NULL, "0", "0", NULL, "2018-07-27 15:06:46.757")"""
% (cusa.sfo['TITLE_ID'], cusa.sfo['CONTENT_ID'], cusa.sfo['TITLE'], cusa.sfo['TITLE_ID'], cusa.size))
print("Completed %d" % cusa.size)
else :
print("Skipped")
if(len(sql_list) > 0) :
cursor.execute("INSERT INTO %s VALUES %s;" % (tbl[0], ', '.join(sql_list)))
print('')
print('')
print('')
print("Processing table: tbl_appinfo")
cursor.execute("SELECT DISTINCT T.titleid FROM (SELECT titleid FROM %s) T WHERE T.titleid LIKE 'CUSA%%' AND T.titleid NOT IN (SELECT DISTINCT titleid FROM tbl_appinfo);" % (" UNION SELECT titleid FROM ".join(tbl_appbrowse)))
missing_appinfo_cusa_id = cursor.fetchall()
for tmp_cusa_id in missing_appinfo_cusa_id :
game_id = tmp_cusa_id[0]
print(" Processing GameID: %s... " % game_id, end='')
cusa = get_game_info_by_id(game_id)
if(cusa.is_usable == True) :
sql_items = appinfo.get_pseudo_appinfo(cusa.sfo, cusa.size)
for key, value in sql_items.items():
cursor.execute("INSERT INTO tbl_appinfo (titleid, key, val) VALUES (?, ?, ?);", [game_id, key, value])
print("Completed")
else :
print("Skipped")
conn.commit()
conn.close()
ftp.cwd('/system_data/priv/mms/')
file = open(app_db,'rb')
ftp.storbinary('STOR app.db', file)
file.close()

 

[Sparkss]

double post

 

[Sparkss]

for the directory to backup to, if running from the NAS then it will be pointing to a local directory, not a share, although that local dir may also have a share associated, the script should not be using that. You need to figure out the path to the directory you want, and if you plan to share the package you will need to do a fair bit of coding to accommodate all the various drive configs (IE: separate drive, JBOD, RAID, etc.). If just for you then that is easy. Let me know which one?

For the IP you can always have it read in a config file that you maintain outside of the package/script. That can contain the IP. Think of it like the exports file for NFS.

EDIT: I have not reviewed the script yet, my comments are just based on your comments from your last post.

 

[Rasscal]

Thnak for the help

I plan on sharing
The NAS spk is created by a windows app that will allow a user to edit the scripts within the folder & run a windows cmd file that will re-pack the spk after the scripts have been edited.

so if the correct configuration for personal use is known i can create a tutorial for this.

If just for you then that is easy. Let me know which one?

so we can start with just for me at the moment

 

[Sparkss]

Thnak for the help

I plan on sharing
The NAS spk is created by a windows app that will allow a user to edit the scripts within the folder & run a windows cmd file that will re-pack the spk after the scripts have been edited.

so if the correct configuration for personal use is known i can create a tutorial for this.



so we can start with just for me at the moment

That makes it easier. I would also recommend putting that folder path into the same config file as the IP address. Then your config script just has to update/create that config file rather than updating the script itself, which is risky and in a professional setting almost never done.

To find the fully qualified path to the directory you want to backup to, open a console (telnet/ssh) to your NAS and navigate to the directory that you want to use. Issue the 'pwd' command (Print Working Directory) and capture the output. That is the path you need to use in your script/config.

If you need a different starting point (IE: you don't know where the directory is in your tree), open a console to the NAS and use the 'mount' command to list all of the file systems mounted. Or another way is to use the disk free command 'df -h' to list all of your file systems w/ free space for each. That output might be easier to read too.

If you are hitting an SMB share from Windows to check on the files, you should be able to go into your NAS's File Manager and view the SMB share properties, including the directory it is mounted on. That should include the full path.

If you are still running into a roadblock you can post the results and we can work from there.

I apologize for not fully reviewing the script. We had a bunch of family stuff this past weekend and now I am back to work. But if you are still running into issues I will try to carve some time out to take a look.