Populate Search Box and then Scrape Web Site for result

Gizzmo · Dec 1st 2020

I am trying to automate the 'Search' box on the following website "https://www.blackwoods.com.au". I have tried to modify original source code from "xl-searchbot.xlsm" from http://automatetheweb.net/", but having no luck in actually getting the 'Search' box to be populated. My understanding of the html and DOM structure is limited and I have being trying for the last 2 days with no luck.

Code

'Original source code from "http://automatetheweb.net/" xl-searchbot.xlsm
Sub ScrapeBlackwoods()
Dim objIE As InternetExplorer 'special object variable representing the IE browser
Dim aEle As Object 'HTMLLinkElement 'special object variable for an <a> (link) element
'initiating a new instance of Internet Explorer and asigning it to objIE
Set objIE = New InternetExplorer
'make IE browser visible (False would allow IE to run in the background)
objIE.Visible = True
'navigate IE to this web page
objIE.Navigate "https://www.blackwoods.com.au" '"search/?text=12345"
'wait here a few seconds while the browser is busy
Do While objIE.Busy = True Or objIE.ReadyState <> 4: DoEvents: Loop
Set aEle = objIE.Document.getElementsByClassName("searchProducts")(0)
'Print to see what is being set and if I am on the right track
Debug.Print aEle.ClassName
Debug.Print aEle.TagName
Debug.Print aEle.ID
'Debug.Print aEle.href
Debug.Print aEle.innerText
Debug.Print aEle.innerHTML
Debug.Print aEle.outerHTML
'in the search box put a value that represents the suppliers product code which eventually will be listed on sheet 1
'for now just enter 12345 as a test
Call aEle.SetAttribute("value", "123456")
Do While objIE.Busy = True Or objIE.ReadyState <> 4: DoEvents: Loop
'close the browser
objIE.Quit
'exit our SearchBot subroutine
End Sub

Display More

This is the HTML Extract for what I believe is the particular area of the DOM that I need to interact with.

Code

<form name="search_form_SearchBox" class="searchProducts" action="/search/" method="get">
<div class="input-group">
<input name="text" class="form-control js-site-search-input searchProductsheader ui-autocomplete-input" role="search" aria-haspopup="true" type="text" maxlength="100" placeholder="Search Products / Category / Brand" value="" data-options='{"autocompleteUrl" : "/search/autocomplete/SearchBox","minCharactersBeforeRequest" : "2","waitTimeBeforeRequest" : "400","displayProductImages" : true}' autocomplete="off">
<span class="input-group-btn"> <button class="clear-icon-search fa fa-close" type="button"></button>
<button class="btn btn-link js_search_button" onclick="SearchResultRedirect1()" type="button" data-rippleria="">
<i class="bw-icon icon-search"></i><span class="header_search_button_text">Search</span>
</button>
</span>
</div>
</form>

Display More

I would aApprecaite any help or advise.

saurabhjaina211 · Dec 29th 2020

Hi Gizzmo,

please use below code:

Code

Sub populateSearchBox()
On Error Resume Next
Dim HTMLDoc As New HTMLDocument
Dim ieBrowser As New InternetExplorer
Dim rowno As Integer
Dim htmlobject As Object
Dim trow As Object
'To open and show Internet Explorer
ieBrowser.Visible = True
'To Open website in Internet Explorer
ieBrowser.navigate "http://automatetheweb.net/"
Do
' Wait till the Browser is loaded
Loop Until ieBrowser.readyState = READYSTATE_COMPLETE
Set HTMLDoc = ieBrowser.document
'<input type="search" class="search-field" placeholder="Search …" value="" name="s">
HTMLDoc.all.s.Value = "123456"
End Sub

Display More

**royUK** · Dec 29th 2020

You both need to read the Forum Rules.

We do not allow questions on web scraping

Quote

5c. Web scraping. Ozgrid will not assist you in systematically acquiring data (even in the public domain) from a other peoples websites. We will show you how to do excel/browser automation with generic examples and answer specific questions related to browser automation VBA methods

Share