To get started with URL Filtering, you must first log on to the Citrix SWG Wizard. The wizard takes you through a series of configuration steps before you apply the URL Filtering policies. Before you begin, be sure you have a valid URL Threat Intelligence feature license installed on your appliance. URL filter configuration settings. If you are editing a URL filter, you are automatically redirected to this page. If you are editing an existing URL filter setting and want to change the name, enter a new name in this field. You must select OK to save the change. In a ZPF configuration, a local URL list can be created for each URL filtering parameter map. You can use Cisco SDM to create list entries and you can import entries from a list stored on your PC. When a local URL list is used in combination with URL filter servers, local entries are used first.
ON THIS PAGE
Web Filtering provides URL filtering capabilityby using either a local Websense server or Internet-based SurfControlserver. For more information, see the following topics:
Enhanced Web Filtering Overview
Enhanced Web Filtering (EWF) with Websense is an integratedURL filtering solution. When you enable the solution on the device,it intercepts the HTTP and the HTTPS requests and sends the HTTP URLor the HTTPS source IP to the Websense ThreatSeeker Cloud (TSC). TheTSC categorizes the URL into one of the 95 or more categories thatare predefined and also provides site reputation information. TheTSC further returns the URL category and the site reputation informationto the device. The device determines if it can permit or block therequest based on the information provided by the TSC.
Starting in JunosOS Release 15.1X49-D40 and Junos OS Release 17.3R1, EWF supports HTTPStraffic by intercepting HTTPS traffic passing through the SRX Seriesdevice. The security channel from the SRXSeries device is divided as one SSL channel between the client andthe SRX Series device and another SSL channel between the SRX Seriesdevice and the HTTPS server. SSL forward proxy acts as the terminalfor both channels and forwards the cleartext traffic to the UTM. UTMextracts the URL from the HTTP request message.
You can consider the EWF solution as the next-generation URLfiltering solution, building upon the existing Surf-Control solution.
Enhanced Web Filtering supports the following HTTP methods:
-
GET
-
POST
-
OPTIONS
-
HEAD
-
PUT
-
DELETE
-
TRACE
-
CONNECT
User Messages and Redirect URLs for Enhanced Web Filtering(EWF) on SRX Series Devices
Starting withJunos OS Release 15.1X49-D110, a new option, custom-objects command that enablesyou to configure user messages and redirect URLs to notify users whena URL is blocked or quarantined for each EWF category. The Name: Name of the custom message; maximum lengthis 59 bytes.
-
user-message
- or Content: Content of the custom message; maximumlength is 1024 bytes.
You configure a user message or redirect URL as a custom objectand assign the custom object to an EWF category.
-
User messages indicate that website access has been blockedby an organization's access policy. To configure a user message, includethe message-text statement at the message] hierarchy level.
-
Redirect URLs redirect a blocked or quarantined URL toa user-defined URL. To configure a redirect URL, include the [edit security utm custom-objects custom-message custom-message option provides the followingbenefits:
-
You can configure a separate custom message or redirectURL for each EWF category.
-
The custom-message configuration option is appliedfor each category. The License keyâThe EWF solutionbuilds upon the SurfControl integrated feature on the device. Twodifferent valid license keys are required for the SurfControl integratedsolution and for EWF. You need to install a new license to upgradeto the EWF solution.You can ignore the warning message 'requires 'wf_key_websense_ewf'licenseâ because it is generated by routine EWF license validationcheck.A grace period of 30 days, consistent with other UTM features,is provided for the EWF feature after the license key expires.The device will continue to support the SurfControl integratedsolution after the upgrade.When the grace period for the EWF feature has passed (or ifthe feature has not been installed), Web filtering is disabled, allHTTP requests bypass Web filtering, and any connections to the TSCare disabled. When you install a valid license, the connections tothe server are established again.
-
The TCP connection between a Web client and awebserverâAn application identification (APPID)module is used to identify an HTTP connection. The EWF solution identifiesan HTTP connection after the device receives the first SYN packet.If an HTTP request has to be blocked, EWF sends a block message fromthe device to the Web client. EWF further sends a TCP FIN requestto the client and a TCP reset (RST) to the server to disable the connection.The device sends all the messages through the flow session. The messagesfollow the entire service chain.
-
HTTPS request interceptionâStarting with Junos OS 15.1X49-D40 and JunosOS Release 17.3R1, EWF intercepts HTTPS traffic passing through theSRX Series device. The security channel from the SRX Series deviceis divided as one SSL channel between the client and the SRX Seriesdevice and another SSL channel between the SRX Series device and theHTTPS server. SSL forward proxy acts as the terminal for both channelsand forwards the cleartext traffic to the UTM. UTM extracts the URLfrom the HTTP request message.
-
JuniperWeb Filtering:Juniper Web Filtering has been set to block this site.CATEGORY: Enhanced_Search_Engines_and_Portals REASON: BY_PRE_DEFINED . However, the corresponding syslog message on the device undertest (DUT) is: .
-
HTTP protocol communication with the TSCâEWF uses the HTTP 1.1 protocol to communicate with the TSC.This ensures a persistent connection and transmission of multipleHTTP requests through the same connection. A single HTTP request orresponse is used for client or server communication. The TSC can handlequeued requests; for optimal performance, an asynchronous requestor response mechanism is used. The requests are sent over TCP, soTCP retransmission is used to ensure request or response delivery.TCP also ensures that valid in-order, non-retransmitted HTTP streamdata is sent to the HTTP client on the device.
-
set securityutm feature-profile web-filtering juniper-enhanced profile juniper-enhancedfallback-settings default ?The response also contains the site categorization and sitereputation information.
-
CachingâSuccessfullycategorized responses are cached on the device. Uncategorized URLsare not cached. The size of the cache can be configured by the user.
-
safe=active. This safe-search string is appended tothe URL, and a redirect response for redirecting the client's querywith safe search is turned on. This ensures that no unsafe contentis returned to the client. If the TSC indicates that it needs to besafe-searched, then you can perform the safe-search redirect.For example, the client makes a request to the URL http://images.example.com/images?hl=en&source=imghp&biw=1183&bih=626&q
=adult+movies&gbv=2&aq=f&aqi=&aql=&oq=&gs_rfai=No category action is defined for this URL. TSC returnssafe-search string NoteSafe-search redirect supports HTTP only. You cannot extractthe URL for HTTPS. Therefore it is not possible to generate a redirectresponse for HTTPS search URLs. Safe-search redirects can be disabledby using the CLI option Site reputationâThe TSCprovides site reputation information. Based on these reputations,you can choose a block or a permit action. If the URL is not handledby a whitelist or a blacklist and does not fall in a user or predefinedcategory, then the reputation can be used to perform a URL filteringdecision.Starting with JunosOS Release 17.4R1, the reputation base scores are configurable. Userscan apply global reputation values, provided by the Websense ThreatSeekerCloud (TSC). For the non-category URLs, the global reputation valueis used to perform filtering,The reputation scores are as follows:-
100-90âSite is considered very safe.
-
80-89âSite is considered moderately safe.
-
70-79âSite is considered fairly safe.
-
60-69âSite is considered suspicious.
-
0-59âSite is considered harmful.
The device maintains a log for URLs that are blocked or permittedbased on site reputation scores. -
-
junos-wf-enhanced-default, is provided to users if they choose not to define their own profile.You can also define an action based on site reputations in aprofile to specify the action when the incoming URL does not belongto any of the categories defined in the profile. If you do not configurethe site reputation handling information, then you can define a defaultaction. All URLs that do not have a defined category or defined reputationaction in their profile will be blocked, permitted, logged-and-permitted,or quarantined depending on the block or permit handling for the defaultaction explicitly defined in the profile. If you do not specify adefault action, then the URLs will be permitted. For search enginerequests, if there is no explicit user-defined configuration, andthe URL request is without the safe-search option, then EWF generatesa redirect response and sends it to the client. The client will generatea new search request with the safe-search option enabled.A URL filtering profile can contain the following items:
-
Multiple user-defined and predefined categories, eachwith a permit or block action
-
Multiple site reputation handling categories, each witha permit or block action
-
One default action with a permit or block action
The order of search is blacklist, whitelist, user-defined category,predefined category, safe-search, site reputation, and default action. -
User Messages and Redirect URLs for Enhanced Web Filtering(EWF) on SRX Series Devices
Starting withJunos OS Release 15.1X49-D110, a new option, custom-objects statement that enablesyou to configure user messages and redirect URLs to notify users whena URL is blocked or quarantined for each EWF category. The Name: Name of the custom message; maximum lengthis 59 ASCII characters. -
-
user-message or Content: Content of the custom message; maximumlength is 1024 ASCII characters.
You configure a user message or redirect URL as a custom objectand assign the custom object to an EWF category.-
User messages indicate that website access has been blockedby an organization's access policy. To configure a user message, includethe message-text statement at the message] hierarchy level.
-
Redirect URLs redirect a blocked or quarantined URL toa user-defined URL. To configure a redirect URL, include the [edit security utm custom-objects custom-message custom-message option provides the followingbenefits:
-
You can configure a separate custom message or redirectURL for each EWF category.
-
The custom-message configuration option is appliedfor each category. The
- Configure UTM custom objects for the UTM features. Setthe interval, set the start time, and enter the URL of category packagedownload:
- Configure the predefined base filters. Each EWF categoryhas a default action in a base filter, which is attached to the userprofile to act as a backup filter. If the categories are not configuredin the user profile, then the base filter takes the action. You canalso upgrade the base filters online.
-
show security utm custom-objectsshow security utm feature-profile web-filtering juniper-enhanced
See also
Example: Configuring Enhanced Web Filtering
This example shows how to configure EnhancedWeb filtering (EWF) for managing website access. This feature is supportedon all SRX Series devices. The EWF solution intercepts HTTP and theHTTPS requests and sends the HTTP URL or the HTTPS source IP to theWebsense ThreatSeeker Cloud (TSC). The TSC categorizes the URL intoone of the 151 or more predefined categories and also provides sitereputation information. The TSC further returns the URL category andthe site reputation information to the device. The SRX Series devicedetermines whether it can permit or block the request based on theinformation provided by the TSC.Requirements
This example uses the following hardware and software components:-
SRX5600 device
-
Junos OS Release 12.1X46-D10 or later
Before you begin, you should be familiar withWeb filtering and Enhanced Web filtering (EWF). See Web Filtering Overview and Understanding Enhanced Web Filtering Process.Overview
Web filtering is used to monitor and control how users accessthe website over HTTP and HTTPS. In this example, you configure aURL pattern list (whitelist) of URLs or addresses that you want tobypass. After you create the URL pattern list, define the custom objects.After defining the custom objects, you apply them to feature profilesto define the activity on each profile, apply the feature profileto the UTM policy, and finally attach the Web filtering UTM policiesto the security policies. Table 1 shows information about EWF configuration type, steps, andparameters used in this example.Table 1: Enhanced Web filtering (EWF) ConfigurationType, Steps, and ParametersConfiguration TypeConfiguration StepsConfiguration ParametersConfigure a URL pattern list (whitelist) of URLs or addressesthat you want to bypass.Create a custom object called urllist3 that contains the patternhttp://www.example.net 1.2.3.4-
[http://www.example.net 1.2.3.4]
-
value urllist3
-
http://www.untrusted.com
-
http://www.trusted.com
Add the urllist3 custom object to the custom URL categorycusturl3.-
urllistblack
-
urllistwhite
Configure the Web filtering feature profile:-
Set the URL blacklist filtering category to custblacklist,set the whitelist filtering category to custwhitelist, and set thetype of Web filtering engine to juniper-enhanced. Then you set thecache size and cache timeout parameters.
-
custwhitelist
-
custblacklist
-
type juniper-enhanced
-
cache size 500
-
cache timeout 1800
-
Name the EWF server and enter the port number for communicatingwith it. (Default port is 80.) Then you create an EWF profile name.
-
rp.cloud.threatseeker.com
-
port 80
-
http-profile my_ewfprofile01
-
Select a category from the included whitelist and blacklistcategories or select a custom URL category list you created for filteringagainst.
-
http-reassemble
-
http-persist
-
Action: log-and-permit
-
site-reputation-action:
-
very-safe permit
-
-
Enter a custom message to be sent when HTTP requests areblocked. Finally, enter a timeout value in seconds.
-
ewf_my_profile-default block
-
custom-block-message '***access denied ***'
-
fallback-settings:
-
server-connectivity block
-
timeout block
-
too-many-requests block
-
-
quarantine-custom-message â**The requested webpageis blocked by your organization's access policy**â.
-
quarantine-message type custom-redirect-url
-
quarantine-message url besgas.spglab.example.net
-
ewf_my_profile-default:
-
timeout 10
-
no-safe-search
-
Configuration
This example shows how to configure custom URLpatterns, custom objects, feature profiles, and security policies.Configuring Enhanced Web Filtering Custom Objects and URL Patterns
CLI Quick Configuration
To quickly configure this section of the example,copy the following commands, paste them into a text file, remove anyline breaks, change any details necessary to match your network configuration,copy and paste the commands into the CLI at the commit from configuration mode.Starting withJunos OS Release 15.1X49-D110, the â -
-
Configure a URL pattern list (whitelist) of URLs or addressesthat you want to bypass. After you create the URL pattern list, youcreate a custom URL category list and add the pattern list to it.Configure a URL pattern list custom object by creating the list nameand adding values to it as follows:Note
Because you use URL pattern lists to create custom URLcategory lists, you must configure URL pattern list custom objectsbefore you configure custom URL category lists.NoteThe guideline to use a URL pattern wildcard is as follows:Use http://. You can use â.â. You can use âhttp://*.http://www.example.ne?, show security utm custom-objects
-
If you are done configuring the device, enter [edit] hierarchylevel, and then enter http-reassemble and show security utm feature-profileweb-filtering command.Step-by-Step Procedure
The following example requires you to navigate variouslevels in the configuration hierarchy. For instructions on how todo that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.To configure the EWF feature profiles:- Configure the Web filtering URL blacklist, URL whitelist,and the Web filtering engine.
- Set the cache size and cache timeout parameters for theconfigured EWF engine.
- Set the server name or IP address and the port numberfor communicating with the server. The default host value in the systemis rp.cloud.threatseeker.com.
- Set the http-persist statement tocheck every HTTP request packet in the same session. If the http-persist statement is not configured for cleartext HTTP traffic, then EWFdoes not check every HTTP request packet in the same session.
- Create a profile name, and select a category from theincluded whitelist and blacklist categories.
- Specify the action to be taken depending on the site reputationreturned for the URL if there is no category match found.
- Enter a custom message to be sent when HTTP requests areblocked.
- Define a redirect URL server so that instead of the devicesending a block page with plain text HTML, the device will send anHTTP 302 redirect to this redirect server with some special variablesembedded in the HTTP redirect location field. These special variablescan be parsed by the redirect server and serve a special block pageto the client with rich images and formatting.
If you configure the security utm feature-profile web-filtering juniper-enhancedprofile ewf_my_profile custom-block-message configuration.
- Specify a default action (permit, log and permit, block,or quarantine) for the profile, when no other explicitly configuredaction (blacklist, whitelist, custom category, predefined categoryactions, or site reputation actions) is matched .
- Configure the fallback settings (block or log and permit)for this profile.
- Enter a timeout value in seconds. When this limit is reached,fallback settings are applied. This example sets the timeout valueto 10. You can also disable the safe-search functionality. By default,search requests have safe-search strings attached to them, and a redirectresponse is sent to ensure that all search requests are safe or strict.Note
The timeout value range for SRX210, SRX220, SRX240, SRX300,SRX320, SRX345, SRX550, SRX1500, SRX4100, and SRX4200 is 0 through1800 seconds and the default value is 15 seconds. The timeout valuerange for SRX3400 and SRX3600 is 1 through 120 seconds and the defaultvalue is 3 seconds.
- Configure a UTM policy (mypolicy) for the Web-filteringHTTP protocol, associating ewf_my_profile to the UTM policy, and attachthis policy to a security profile to implement it.
Results
From configuration mode, confirm your configurationby entering the commit from configuration mode.Attaching Web Filtering UTM Policies to Security Policies
CLI Quick Configuration
To quickly configure this section of the example,copy the following commands, paste them into a text file, remove anyline breaks, change any details necessary to match your network configuration,copy and paste the commands into the CLI at the commit from configuration mode.Step-by-Step Procedure
The following example requires you to navigate variouslevels in the configuration hierarchy. For instructions on how todo that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.To attach a UTM policy to a security policy:- Create the security policy sec_policy.
- Specify the match conditions for sec-policy.
- Attach the UTM policy mypolicy to the security policysec_policy.
Results
From configuration mode, confirm your configurationby entering the commit from configuration mode.Verification
To confirm that the configuration is workingproperly, perform these tasks:Verifying the Status of the Web Filtering Server
Purpose
Verify the Web filtering server status.Action
From the top of the configuration in operational mode,enter the show security utm web-filtering statistics command.user@host> show securitypolicy
-
user@host> Quarantine Message
The quarantine message sent to the HTTP client is user-configurableand is of the following types:
-
Default message
The default quarantine message is displayed when a user attemptsto access a quarantined website and it contains the following information:
-
URL name
-
Quarantine reason
-
Category (if available)
-
Site-reputation (if available)
For example, if you have set the action for Enhanced_Search_Engines_and_Portalsto quarantine, and you try to access www.search.example.com,the quarantine message is as follows:
-
-
Syslog message.
The syslog message will be logged by the system when the useraccess the web page that has already been quarantined and marked asblock or permit.
The corresponding syslog message on the device under test is:
Starting in JunosOS 12.1X47-D40 and Junos OS Release 17.3R1, the structured log fieldshave changed. The structured log field changesin the UTM Web filter logs WEBFILTER_URL_BLOCKED, WEBFILTER_URL_REDIRECTED,and WEBFILTER_URL_PERMITTED are as follows:
-
error-message -> reason
-
object-name -> url
-
custom-message, is added for the custom-message option has the following mandatoryattributes:
-
Type: Type of custom message: redirect-url.
-
message-text statement at the message] hierarchy level.
-
Redirect URLs redirect a blocked or quarantined URL toa user-defined URL. To configure a redirect URL, include the [edit security utm custom-objects custom-message custom-message option provides the followingbenefits:
-
Free dance ejay download. You can configure a separate custom message or redirectURL for each EWF category.
-
The The requested webpage is blocked by your organization's access policy.
You block URLs in the Enhanced_News_and_Media category and permitURLs in the Enhanced_Education category. Then you quarantine the URLsin the Enhanced_Streaming_Media category and configure the deviceto send the following message: .
In this example, you set the default action to permit. You selectfallback settings (block or log-and-permit) for this profile in caseerrors occur in each configured category. Finally, you set the fallbacksettings to block.
Configuration
Configuring Site Reputation Action
CLI Quick Configuration
To quickly configure this example, copy thefollowing commands, paste them into a text file, remove any line breaks,change any details necessary to match your network configuration,copy and paste the commands into the CLI at the commit from configuration mode.
Step-by-Step Procedure
The following example requires you to navigate variouslevels in the configuration hierarchy. For instructions on how todo that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.
To configure the site reputation action:
Configure the Web Filtering URL whitelist.
Specify the Enhanced Web Filtering engine, and set thecache size parameters.
-
Configure the base reputation scores. Note
The base reputation value must be ordered.
Set the cache timeout parameters.
Create a profile name, and select a category from thewhitelist categories.
Create a profile name, and select a category from thewhitelist categories.
Enter a warning message to be sent when HTTP requestsare quarantined.
Select a default action (permit, log-and-permit, block,or quarantine) for the profile, when no other explicitly configuredaction (blacklist, whitelist, custom category, predefined categoryor site reputation ) is matched .
Select fallback settings (block or log-and-permit) forthis profile.
Results
From configuration mode, confirm your configurationby entering the commit from configuration mode.
Verification
Confirm that the configuration is working properly.
Verifying the Status of UTM Service
Purpose
Verify the UTM service status.
Action
From operational mode, enter the Automatic Proxy Configuration Url List
Action
From operational mode, enter the show securityutm web-filtering status command.
Sample Output
Verifying the Statistics of UTM Web Filtering
Purpose
Verify the Web filtering statistics for connectionsincluding whitelist and blacklist hits and custom category hits.
Action
From operational mode, enter the show security utm web-filtering statistics
-
-
-
See also
SRX TAP Mode Support Overview
The TAP (Terminal Access Point) mode is a standby device,which checks the mirrored traffic through switch. If UTM is enabled,then the TAP mode inspects the incoming and outgoing traffic by configuringthe TAP interface and generating a security log report to show thenumber of threats detected and the user usage. If some packet getslost in the tap interface, the UTM terminates the connection, as aresult no report is generated for this connection. The UTM configurationremains the same as non-TAP mode.
Starting in Junos OS Release 19.1R1, a TAP (Terminal AccessPoint) mode is supported on the UTM module. When you configure theSRX Series device to operate in TAP mode, the device generates securitylog information to display the information on threats detected, applicationusage, and user details.
when configured to operatein TAP mode, the SRX Series device receives packets only from theconfigured TAP interface.
Note
when configured to operatein TAP mode, the SRX Series device receives packets only from theconfigured TAP interface.
You can configure only one interface to operate in TAPmode.
Crack Url Filter Configuration Download
UTM functionality configured on SRX Series device continuesto work and exchange information from server as per configuration.To use UTM functionality when the SRX Series device is configuredin TAP mode, you must configure the DNS server to resolve the cloudserverâs IP addresses.
The connection between SRX device and Ethernet switch is a mirrorconnection for the connection between client and Ethernet switch.The mirror port allows copying of traffic on the switch. When youconfigure an interface on the SRX Series device to operate as tapmode interface and connecting it with a switch, the switch mirrorport provides the SRX Series device with the mirrored traffic. SRXSeries device process the incoming traffic from one TAP interfaceand generates security log information to display the informationon threats detected, application usage, and user details.
When operating in TAP mode, the SRX Series device performs:
-
Enhanced Web filtering (EWF) for mirrored HTTP traffic.
-
Sophos antivirus (SAV) for mirrored HTTP/FTP/SMTP/POP3/IMAPtraffic.
-
Antispam (AS) for mirrored SMTP traffic.
Related Documentation
Description
Starting with JunosOS Release 17.4R1, support for custom category configuration is availablefor local and Websense redirect profiles.
Starting with JunosOS Release 17.4R1, you can download and dynamically load new EWF categories.The downloading and dynamic loading of the new EWF categories do notrequire a software upgrade. Websense occasionally releases new EWFcategories. EWF classifies websites into categories according to host,URL, or IP address and performs filtering based on the categories.
Starting with JunosOS Release 17.4R1, predefined base filters, defined in a categoryfile, are supported for individual EWF categories. Each EWF categoryhas a default action in a base filter, which is attached to the userprofile to act as a backup filter. If the categories are not configuredin the user profile, then the base filter takes the action.
Starting with JunosOS Release 17.4R1, the reputation base scores are configurable. Userscan apply global reputation values, provided by the Websense ThreatSeekerCloud (TSC). For the non-category URLs, the global reputation valueis used to perform filtering,
Starting with JunosOS Release 17.4R1, support for custom category configuration is availablefor local and Websense redirect profiles.
Starting with JunosOS Release 17.4R1, support for custom category configuration is availablefor local and Websense redirect profiles.
Starting in JunosOS Release 15.1X49-D40 and Junos OS Release 17.3R1, EWF supports HTTPStraffic by intercepting HTTPS traffic passing through the SRX Seriesdevice.
Starting with Junos OS 15.1X49-D40 and JunosOS Release 17.3R1, EWF intercepts HTTPS traffic passing through theSRX Series device. The security channel from the SRX Series deviceis divided as one SSL channel between the client and the SRX Seriesdevice and another SSL channel between the SRX Series device and theHTTPS server. SSL forward proxy acts as the terminal for both channelsand forwards the cleartext traffic to the UTM. UTM extracts the URLfrom the HTTP request message.
Starting withJunos OS Release 15.1X49-D110, a new option, custom-objects command that enablesyou to configure user messages and redirect URLs to notify users whena URL is blocked or quarantined for each EWF category.
Starting withJunos OS Release 15.1X49-D110, a new option, custom-objects statement that enablesyou to configure user messages and redirect URLs to notify users whena URL is blocked or quarantined for each EWF category.
Starting withJunos OS Release 15.1X49-D110, the âcustom-message, is added for the custom-objects statement that enablesyou to configure user messages and redirect urls to notify users whena url is blocked or quarantined for each ewf category.
starting with junosos release 12.3x48-d25 and junos os release 17.3r1, enhanced web filtering(ewf) over ssl forward proxy supports https traffic.
starting in junosos 12.1x47-d40 and junos os release 17.3r1, the structured log fieldshave changed.
>