Webread Query Not Working

13 views (last 30 days)
Eric Rapos
Eric Rapos on 30 May 2019
Answered: Kanishk on 19 Aug 2024
I am looking to query a forum style page (this forum, actually), but cannot seem to the the queries working as desired.
My goal is to essentially get the most viewed questions about Simulink (as woud be done via this page: https://www.mathworks.com/matlabcentral/answers/?product_base_code%5B%5D=Simulink&sort=views+desc)
However, when I try this via webread, the page I get is just the base Answers page, and it isn't giving me Simulink, nor the sorting.
I am using the following (where indexed will be looped for multiple pages):
url = 'https://www.mathworks.com/matlabcentral/answers/';
data = webread(url,'page', index,'product_base_code%5B%5D','Simulink','sort','views+desc');
Any advice/help would be greatly appreciated.
Thanks!

Answers (1)

Kanishk
Kanishk on 19 Aug 2024
Hi Eric,
I understand you're attempting to access the MATLAB forums page using the ‘webread’ function with specific query parameters. However, directly utilizing the URL for the desired page with ‘webread results in the following error.
'''
The connection to URL 'https://www.mathworks.com/matlabcentral/answers/?product_base_code%5B%5D=Simulink&sort=views+desc' timed out after 5.000 seconds. The reason is "Waiting for response header". Perhaps the server is not responding or weboptions.Timeout needs to be set to a higher value.
'''
Increasing the ‘Timeout’ value using weboptions’ as suggested in the error works and fetches the correct page. Here is the code for same.
webopts = weboptions('Timeout', 10);
data = webread('https://in.mathworks.com/matlabcentral/answers/?page=1&sort=views+desc&term=product_base_code%3ASL', webopts);
To incorporate a page index into the URL within a for loop, you can utilize the ‘strcat’ function.
for index = 1:5
webopts = weboptions('Timeout', 10);
url = strcat('https://in.mathworks.com/matlabcentral/answers/?page=',num2str(index),'&sort=views+desc&term=product_base_code%3ASL');
data = webread(url, webopts);
end
The above code will fetch 5 pages from MATLAB central. You can use ‘websave’ if you wish to save the fetched pages in HTML files.
Please go through following MATLAB documentation to learn more about ‘weboptions, strcat and ‘websave’:
Hope This helps!
Thank You
Kanishk Singhal

Tags

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!