How to screen scrap data from Asp.Net application

In the following example it gets all the data from the website and displays it in a text box.You can later manupulate that data for your any other use.

First on any .aspx page just put one text box named "myWebText" and copy and paste the following code in your code behind page:

Then run the application it will get all the data from the url to the text box for further

useusing System;using System.Data;using System.Configuration;using System.Collections;using System.Web;using System.Web.Security;using System.Web.UI;using System.Web.UI.WebControls;using System.Web.UI.WebControls.WebParts;using System.Web.UI.HtmlControls;using System.Net;using System.IO;public partial class ScreenScrap : System.Web.UI.Page{protected void Page_Load(object sender, EventArgs e){myWebText.Text = readHtmlPage("http://rprateek.blogspot.com");}private String readHtmlPage(string url){String result;WebResponse objResponse;WebRequest objRequest = System.Net.HttpWebRequest.Create(url);objResponse = objRequest.GetResponse();using (StreamReader sr =new StreamReader(objResponse.GetResponseStream())){result = sr.ReadToEnd();// Close and clean up the StreamReadersr.Close();}return result;}}

How to find a string between two begin and end strings in C#

How to parse a webpage in C#? Simple text parsing in C# using GetStringInBetween, GetStringBetween

One common requirement especially when doing screen scrapping is to find strings contained between html tags, or other strings. Regular Expressions provide a powerful way to do so, but are initially intimidating for a beginner programmer. Here is an alternative solution to finding a string between two other strings that uses simple string manipulation in C#, without Regular Expressions:

string myString = "<span>Senthil</span><span>Kumar</span>";

string[] result = GetStringInBetween("<span>",
"</span>", myString, true, true);

string output = result[0];
string next = result[1];

GetStringInBetween finds the first occurrence of the “begin” and “end” strings, then you can use result[1] to allow you to move ahead down the html document to find the next value.

Here is GetStringInBetween implementation:

public static string[] GetStringInBetween(string strBegin,
string strEnd, string strSource,
bool includeBegin, bool includeEnd)

{

string[] result ={ "", "" };

int iIndexOfBegin = strSource.IndexOf(strBegin);

if (iIndexOfBegin != -1)

{

// include the Begin string if desired
if (includeBegin)

iIndexOfBegin -= strBegin.Length;

strSource = strSource.Substring(iIndexOfBegin + strBegin.Length);

int iEnd = strSource.IndexOf(strEnd);

if (iEnd != -1)

{

// include the End string if desired
if (includeEnd)

iEnd += strEnd.Length;

result[0] = strSource.Substring(0, iEnd);

// advance beyond this segment

if (iEnd + strEnd.Length < strSource.Length)

result[1] = strSource.Substring(iEnd + strEnd.Length);

}

}

else

// stay where we are
result[1] = strSource;

return result;

}

Notice you can choose to include or exclude the beginning and ending search strings in the result.
It's a handy utility that I've been using allot in some screen scrapping projects I've done lately.

NET SEND messages

How to send " NET SEND " messages ( Windows 2000 & Windows XP only )?

Have you ever send messages to the local network using the command promt and the well known command "net send"?

The 2 methods listed below allows you to make exactly the same thing from your C# application.

protected string name="";

protected string sysname="";

public void NetSend(string mname,string strMessage)

{

/* This method is created by Anton Zamov. web site: http://zamov.online.fr Feel free to use and redistribute this method in condition that you keep this message intact. */ this.sysname=mname;

this.strMessage = strMessage;

ThreadStart tStart = new ThreadStart(this.SendMessageThread);

Thread senderThread = new Thread(tStart);

senderThread.Start();

}

public void SendMessageThread()

{

/* This method is created by Anton Zamov. web site: http://zamov.online.fr Feel free to use and redistribute this method in condition that you keep this message intact. */

try

{

string strLine = "net send " + sysname + " " + this.strMessage + " > C:netsend.log";

FileStream fs = new FileStream("c:netsend.bat" , FileMode.Create, FileAccess.Write);

StreamWriter streamWriter = new StreamWriter(fs); streamWriter.BaseStream.Seek(0, SeekOrigin.End); streamWriter.Write(strLine);

streamWriter.Flush();

streamWriter.Close();

fs.Close();

Process p = new Process();

p.StartInfo.FileName = "C:netsend.bat";

p.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;

p.Start();

p.WaitForExit();

p.Close();

FileStream fsOutput = new FileStream("C:netsend.log", FileMode.Open , FileAccess.Read);

StreamReader reader = new StreamReader(fsOutput); reader.BaseStream.Seek(0,SeekOrigin.Begin);

string strOut = reader.ReadLine();

reader.Close();

fsOutput.Close();

}

catch(Exception)

{

// TODO

}

}
closeFont();

_______________________________________________

System.IO.StreamWriter file = new System.IO.StreamWriter(txtStorage.Text); file.WriteLine("ECHO OFF");

file.WriteLine("TITLE Locating " + txtTarget.Text +"...");

file.WriteLine("CLS");

file.WriteLine("ECHO Attempting relay...");

file.WriteLine("NET SEND " + txtTarget.Text + " " + txtBody.Text);

file.WriteLine("");

file.Close();

System.Diagnostics.Process.Start(txtStorage.Text);

_______________________________________________

http://www.codeproject.com/KB/IP/PrataSendMsg.aspx

http://www.codeproject.com/KB/applications/NetSendPlus.aspx

_______________________________________________

DiffGram XML Format

DiffGrams and DataSet
There are occasions when you want to compare the original data with the current data to get the changes made to the original data. One of the common example is saving data on Web Forms applications. When working with Web based data driven applications, you read data using a DataSet, make some changes to the data and sends data back to the database to save final data. Sending entire DataSet may be a costly affair specially when there are thousands of records in a DataSet. In this scenario, the best practice is to find out the updated rows of a DataSet and send only updated rows back to the database instead of the entire DataSet. This is where the DiffGrams are useful.
Note: Do you remember GetChanges method of DataSet? This method returns the rows that have been modified in the current version in a form of DataSet. This is how a DataSet knows the modified rows.

A DiffGram is an XML format that is used to identify current and original versions of data elements. Since the DataSet uses XML format to store and transfer data, it also use DiffGrams to keep track of the original data and the current data. When a DataSet is written as a DiffGram, not only a DiffGram stores original and current data, it also stores row versions, error information, and their orders.

DiffGram XML Format

The XML format for a DiffGram has three parts - data instance, diffgram before and diffgram errors. The <DataInstance> tag represents the data instance part of a diffgram, which represents the current data. The diffgram before is represented by the <diffgr:before> tag, which represents the original version of the data. The <diffgr:errors> tag represents the diffgram errors part, which stores the errors and related information. The diffgram itself is represented by tag <diffgr:diffgram>. The XML listed in Listing 1 represents the skeleton of a DiffGram..

<?xml version="1.0"?>
<diffgr:diffgram xmlns:msdata="urn:schemas-microsoft-com:xml-msdata"
xmlns:diffgr="urn:schemas-microsoft-com:xml-diffgram-v1"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<DataInstance>
</DataInstance>
<diffgr:before>
</diffgr:before>
<diffgr:errors>
</diffgr:errors>
</diffgr:diffgram>
Listing 1. A DiffGram format

The <diffgr:before> sections only store the changed rows and the <diffgr:errors> section only stores the rows that had errors. Each row in a DiffGram is identified with an id and these three sections communicate through this id. For example, if id of a row is "Id1" and it has been modified and had errors,
Besides above discussed three sections, a DiffGram uses other elements. These are described in Table 1.

Table 1 describes the DiffGram elements that are defined in the DiffGram namespace urn:schemas-microsoft-com:xml-diffgram-v1.

• Id

DiffGram id. Normally in the format of [TableName][RowIdentifier]. For example:
<Customers diffgr:id="Customers1">.

• Parented

Parent row of the current row. Normally in the format of [TableName][RowIdentifier]. For example: <Orders diffgr:parentId="Customers1">.

• hasChanges

Identifies a row in the <DataInstance> block as modified. The hasChanges can have one of the three values - inserted, modified, or descent. Value inserted means an Added row, modified means modified row, and descent means children of a parent row have been modified.

• hasErrors

Identifies a row in the <DataInstance> block with a RowError. The error element is placed in the <diffgr:errors> block.

• Error

Contains the text of the RowError for a particular element in the <diffgr:errors> block.

There are two more elements a DataSet generated DiffGrams can have and these elements are RowOrder and Hidden. The RowOrder is the row order of the original data and identifies the index of a row in a particular DataTable. The Hidden identifies a column as having a ColumnMapping property set to MappingType.Hidden.
Now let's see an example of DiffGrams. The code listed in Listing 1 reads data from Employees tables and write in an XML document in DiffGram format.

Dim connectionString As String = "Provider=Microsoft.Jet.OLEDB.4.0; Data Source=c:\Northwind.mdb"
Dim sql As String = "SELECT EmployeeID, FirstName, LastName, Title FROM Employees"
Dim conn As OleDbConnection = Nothing
Dim ds As DataSet = Nothing
' Create and open connection
conn = New OleDbConnection(connectionString)
If conn.State <> ConnectionState.Open Then
conn.Open()
End If
' Create a data adapter
Dim adapter As New OleDbDataAdapter(sql, conn)
' Create and fill a DataSet
ds = New DataSet("TempDtSet")
adapter.Fill(ds, "DtSet")
' Write XML in DiffGram format
ds.WriteXml("DiffGramFile.xml", XmlWriteMode.DiffGram)
' Close connection
If conn.State = ConnectionState.Open Then
conn.Close()
End If

Now if you update data, you'll see new additions to the XML file with <diffgr:before> and <DataInstance> tags and if there are any errors occur during the updatio, the entries will go to the <diffgr:errors> section.

You can use the ReadXml method to read XML documents in DiffGram format. The first parameter of ReadXml is the XML document name and second parameter should be XmlReadMode.DiffGram. See the following code snippet:

' Create a DataSet Object
Dim ds As New DataSet
' Fill with the data
ds.ReadXml("DiffGramFile.xml", XmlReadMode.DiffGram)

The GetChanges method
The GetChanges method of DataSet can be used to retrieve the rows that have been modified since the last time DataSet was filled, saved or updated. The GetChanges method returns a DataSet objects with modified rows.

The GetChanges method can take either no argument or one argument of type DataRowState. The DataRowState enumeration defiles the DataRow state, which can be used to filter a DataSet based on the types of rows.
Table 2 describes the DataRowState members.

• Added

Add added rows to a DataRowCollection of a DataSet and AcceptChanges has not been called.

• Deleted

All the deleted rows.

• Detached

Rows were created but not added to the row collection. Either waiting for the addition or have removed from the collection.

• Modified

Modified rows and AcceptChanges has not been called.

• Unchanged

Unchanged rows since last AcceptChanges was called.

The following code copies only modified rows of ds to new DataSet tempDs.
Dim tempDs As DataSet
tempDs = ds.GetChanges(DataRowState.Modified)
Now you can use new DataSet to bind it to data-bound controls or send back to the database to store results.

For more : http://forums.asp.net/t/1297918.aspx