login to website with curl and download webpage

i am trying to log-in into a website and download a webpage.do not be alarmed- the website and credentials are public knowledge...

anyway, my code is as follows:

curl -k > 01.txt
sfk xed 01.txt "_[start][0.10000 bytes]csrftoken__" +xed "_/>**__" +xed "_\q id=\qcsrftoken\q value=\q__" +xed "_\q __" -tofile 02.txt
set /p csrf=<02.txt
curl -k -d "username=retalix&password=12345&csrftoken=%csrf%" > RL.txt

what the code does is download the login page, extract the csrf variable and use it, plus the username/password, to log-in another sub-"domain" in the same website.

the end result should be a webpage that is saved into "RL.txt" but none of the solutions i could find worked and i can't understand why.

any help would be appreciated!

1 Answer

As I was working on something similar at the time I quickly put this together for you:

Please change the path and filename to your requirements, make sure that the path has a trailing slash.

$path = "c:\test\"
$filename = "website.txt"
$url = ""
$url2 = ""
$ie = New-Object -com InternetExplorer.Application
$ie.visible = $true
$ie.silent = $false
$ie.navigate("$url")
while($ie.ReadyState -ne 4) {start-sleep -m 100}
if ($ie.document.url -Match "invalidcert"){ $sslbypass=$ie.Document.getElementsByTagName("a") | where-object {$_.id -eq "overridelink"} $sslbypass.click() start-sleep -s 5
}
$ie.Document.IHTMLDocument3_getElementById("username").click()
$ie.Document.IHTMLDocument3_getElementById("username").value ='retalix'
$ie.Document.IHTMLDocument3_getElementById("password").click()
$ie.Document.IHTMLDocument3_getElementById("password").value ='12345'
$ie.Document.IHTMLDocument3_getElementById("submit").click()
start-sleep 5
$ie.navigate($url2)
start-sleep 10
$ie.Document.body.outerHTML | Out-File -FilePath $path$filename

This takes around 25 seconds to run due to the start-sleeps, this is to allow the pages to load. Depending on your connection you may need to increase these.

After you have ran the script and are happy with the outcome you can then do the following.

Change the following so that IE does not open up on the desktop:

$ie.visible = $true

to

$ie.visible = $false

and add the following at the end of the script to ensure IE closes at the end:

$ie.quit()

Hope this helps.